Scale of Microsoft Office 365 LLM Serving Revealed Microsoft examined its Office 365 LLM deployment handling more than 10 million daily requests across several data‑center regions, identifying a mix of latency‑sensitive and latency‑insensitive tasks and a variety of SLA requirements [1]. The analysis covered request patterns over multiple weeks, exposing peak loads that strain fast‑task GPU pools while slower tasks occupy idle capacity [1]. These findings form the empirical basis for the proposed cost‑saving system [1].

Current GPU Allocation Practices Lead to Wasted Capacity Existing serving architectures separate fast and slow workloads into distinct GPU pools, causing substantial under‑utilization because the fixed allocations rarely match real‑time demand [1]. Idle accelerators persist during off‑peak periods, inflating operational expenses without improving performance [1]. The study quantifies this inefficiency as a major target for optimization [1].

SageServe Introduces Dynamic Multi‑Timescale Resource Management The new framework routes incoming requests to the most appropriate data center in the short term while simultaneously scaling GPU virtual machines and repositioning models over longer horizons [1]. It relies on traffic forecasts and an Integer Linear Programming optimizer to balance cost and latency objectives [1]. This multi‑timescale control enables rapid adaptation to workload fluctuations [1].

Evaluation Demonstrates Substantial GPU‑Hour Reductions Simulations and live trials on 10 million production requests across three regions and four open‑source models achieved up to 25 % fewer GPU‑hours compared with the baseline deployment [1]. The results maintained tail‑latency SLAs, confirming that cost cuts did not compromise service quality [1]. The evaluation validates SageServe’s potential for large‑scale cloud operators [1].

Auto‑Scaling Optimization Cuts Waste and Saves Millions By eliminating inefficient auto‑scaling behavior, SageServe reduced GPU‑hour waste by 80 %, translating into an estimated $2.5 million monthly cost reduction [1]. The framework preserves performance guarantees while dramatically lowering excess capacity [1]. These savings illustrate the financial impact of smarter resource orchestration [1].

Study Provides Rare Public Insight Into Internet‑Scale LLM Workloads This research represents one of the first publicly available characterizations of Internet‑scale LLM serving, offering data that cloud providers worldwide can leverage for their own optimizations [1]. The authors emphasize the broader relevance of their methodology beyond Microsoft’s internal environment [1]. The paper sets a benchmark for future academic and industry analyses of large‑scale AI inference [1].

Redundant Context Processing Slows Multi‑LLM Pipelines Large language model pipelines increasingly chain several fine‑tuned variants derived from a common base, but each model recomputes the full context during the prefill stage, creating significant latency and throughput bottlenecks [1]. The duplicated work grows linearly with the number of variants, limiting real‑time applications that rely on rapid multi‑LLM responses [1]. Researchers identified this inefficiency as the primary motivation for a new sharing framework [1].

DroidSpeak Reuses KV‑Cache Across Related Models The system inspects the key‑value (KV) cache of the foundational model and isolates layers whose activations remain useful for downstream fine‑tuned versions [1]. For each variant, only the identified layers are recomputed, while the rest of the cache is retained, eliminating redundant computation [1]. This selective reuse targets models that share the same architecture and base weights, enabling seamless integration into existing serving stacks [1].

Selective Layer Recalculation Preserves Accuracy Experiments on diverse datasets show that the layer‑wise caching strategy incurs only a few percentage points deviation from baseline task performance [1]. Accuracy metrics remain within acceptable margins, confirming that speed gains do not come at the cost of significant quality loss [1]. The authors report that the trade‑off is consistent across multiple model pairs and tasks [1].

Benchmarks Show Up to Threefold Throughput Gains On benchmark workloads, DroidSpeak delivers up to a 3× increase in overall inference throughput compared with full recomputation [1]. Prefill latency improves on average by a factor of 2.6, accelerating the initial token generation phase that typically dominates response time [1]. The paper, authored by Shan Lu, Madan Musuvathi, and Esha Choukse, was published in Microsoft Research’s archive on May 1, 2026 [1].

Friday’s Temperatures Reach Seasonal Highs Across Region Friday will be partly sunny with mid‑50s to low‑60s highs; inland fog clears while coastal areas stay cloudy with light sprinkles. Seattle is forecast at 57 °F, Quillayute sets a new 66 °F record, and Hoquiam ties its 61 °F record from 1987. [1]

Saturday Brings Rain and Dropping Snow Level Saturday sees rain intensify in the morning and temperatures rise to the mid‑50s, with low‑40s lows expected. The snow line drops to about 6,000 feet, opening the possibility of mountain snowfall. [1]

Snow Potential Expands Through Early Week Sunday continues with intermittent rain and low‑50s highs, and the snow level falls further to roughly 4,500 feet, increasing chances of snow at higher elevations. Monday through Wednesday bring lingering showers, daytime highs in the upper 40s to 50 °F, and a snow line between 2,500 and 3,000 feet, making snow likely on most passes. [1]

Midweek Dry Spell Followed by Thursday Showers Thursday clears as clouds retreat, leaving partly sunny skies with a few late‑day showers but overall dry conditions. Temperatures climb to the mid‑50s with lows in the low 40s, and the snow level stabilizes near 3,000 feet. [1]

Precipitation Totals May Affect Super Bowl and Olympics Rainfall totals for the week range from 0.5 to 1.75 inches across Puget Sound. Mountain passes could accumulate 2‑6 inches of snow, exceeding 10 inches above 5,000 feet. KING 5 will monitor these conditions for potential impacts on Super Bowl Sunday in Santa Clara and the Winter Olympics in Italy. [1]

Workshop Announcement and Publication Details The “What does Generative UI mean for HCI Practice?” workshop will appear in the Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, officially dated April 1, 2026 [1]. It is scheduled as part of CHI 2026, the premier annual gathering for human‑computer interaction research. The announcement positions the workshop as a focal point for emerging AI‑driven interface discussions.

Organizers and Leadership The event is coordinated by three senior researchers: Siân Lindley, Jack Williams, and Abigail Sellen, who are listed as authors and primary organizers [1]. Their involvement signals strong academic backing and aligns the workshop with ongoing HCI scholarship. Each organizer brings expertise in design, AI, and user experience, shaping the workshop’s agenda.

Scope and Objectives of the Workshop The workshop aims to explore how generative UI technologies can underpin innovative, human‑centric experiences and to identify necessary evolutions in HCI practice [1]. Participants are invited to envision future interface paradigms and assess implications for design methodology. The focus on AI‑generated interfaces reflects growing interest in automating UI creation while preserving usability.

Interactive Format, Submission Options, and Participant Cap Sessions will include a pop‑up panel, creative ideation exercises, and collaborative artefact development, with outcomes shared online and potentially expanded into an Interactions or CACM article [1]. Prospective attendees may submit a two‑page position paper, a two‑page pictorial, or a two‑minute video via the workshop website. Organizers anticipate roughly 35 participants, limiting the event to a focused cohort.

Resources and Future Dissemination The announcement provides direct links to the workshop’s publication page and a downloadable PDF for interested scholars [1]. These resources facilitate early engagement and allow contributors to prepare submissions. The planned artefact sharing and possible journal extensions aim to extend the workshop’s impact beyond the conference.

New Framework Targets Heterogeneous AI Inference Systems The paper “MSCCL++: Rethinking GPU Communication Abstractions for AI Inference” proposes a redesign of GPU data‑exchange mechanisms to boost inference performance on modern heterogeneous hardware, and it was released on March 1, 2026 [1]. It lists six contributors—Changho Hwang, Peng Cheng, Roshan Dathathri, Abhinav Jangda, Madan Musuvathi, and Aashaka Shah—reflecting a cross‑disciplinary effort within Microsoft Research [1].

Authors Highlight Limitations of Existing Communication Libraries Researchers note that AI workloads now depend on a mix of accelerators and CPUs, but current general‑purpose libraries cannot keep pace with rapid hardware evolution [1]. Developers frequently resort to hand‑crafted communication stacks that deliver speed yet introduce bugs and hinder portability across GPU generations [1]. This fragmentation motivates the need for a more adaptable solution.

MSCCL++ Promises Portable Performance Matching Hand‑Crafted Stacks The proposed library rethinks communication primitives to provide abstractions that are both hardware‑agnostic and capable of matching the speed of custom stacks [1]. By eliminating error‑prone bespoke code, MSCCL++ aims to improve robustness while preserving throughput on diverse GPU architectures [1].

Research Presented at Premier Architecture Conference The work was peer‑reviewed and presented at ASPLOS 2026, the ACM International Conference on Architectural Support for Programming Languages and Operating Systems [1]. Inclusion in this venue underscores the significance of the communication challenges for AI inference and the community’s interest in portable solutions.