This year's Microsoft Build conference highly focused on generative AI and teamed up with OpenAI to create a big universe.

In recent months, Microsoft has been busy building generative AI into many of its products and services, including search engine Bing, browser Edge, GitHub and the Office productivity suite.

At the Microsoft Build 2023 conference held early this morning, these results finally came out, and went one step further: Microsoft's AI universe is getting more and more complete.

The topic of this year's Microsoft Build conference is highly focused on AI. Microsoft CEO Satya Nadella first discussed the platform shifts over the years and recalled technological changes, and then directly announced: "We announced more than 50 updates for developers, from bringing Bing to ChatGPT to Windows Copilot to The new Copilot Stack with general scalability, Azure AI Studio, and the new data analytics platform Microsoft Fabric.”

Next, Microsoft released these important updates in succession. Changes that are more relevant to ordinary users include: the addition of the CoPilot experience in Windows 11 and Edge, and the new Bing AI and Copilot plugins for OpenAI ChatGPT.

Copilot comes to Windows 11

This time Microsoft released Windows Copilot, becoming the first PC platform to provide users with centralized AI assistance. Windows Copilot works with Bing Chat and 1st/3rd party plugins, allowing users to focus on bringing their ideas to life, completing complex projects and collaborating instead of finding, launching and working across multiple applications.

Windows Copilot is very simple to invoke and use. When opened, Windows Copilot becomes a sidebar tool integrated into the operating system to help users complete various tasks.

Microsoft says that Windows Copilot will be an efficient personal assistant that helps users take action, customize settings, and seamlessly link to their favorite applications.

In addition, many functions of Windows, including copy/paste, Snap Assist, screenshots and personalization, can be made better through Windows Copilot. For example, users can not only copy/paste, but also ask Windows Copilot to rewrite, summarize or explain the content.

At the same time, just like using Bing Chat, users can ask Windows Copilot a series of simple to complex questions. For example, if I want to call my family in Cyprus, I can quickly check the local time to make sure I don't wake them up in the middle of the night. If I want to plan a trip to visit them in Cyprus, I can ask Windows Copilot to look up flights and accommodation.

Microsoft said that Windows Copilot will begin public testing in June, and then roll it out to existing Windows 11 users.

ChatGPT default Bing search

Since Bing announced access to ChatGPT, Bing users have engaged in more than 500 million chats, created more than 200 million images using Bing Image Creator, and daily downloads of the Bing mobile app have grown 8x since launch.

At this conference, Bing ushered in three key updates: integrating Bing search into ChatGPT, establishing a common plugin platform with OpenAI and new plugin partners, and expanding the integration of Bing Chat throughout Microsoft Copilot.

"Today's update creates greater opportunities for developers and even more amazing experiences for people as we continue to transform search."

One of the notable changes is that Bing will become the default search engine plugin for ChatGPT. Nadella also showed how it would work:

"This is just the beginning of our plan to bring the best of Bing to the ChatGPT experience with our partners at OpenAI," Nadella said.

Microsoft said that after ChatGPT is integrated into the new Bing, it can provide more timely and updated answers through web access. ChatGPT Plus subscribers can experience this feature immediately.

Default Bing search in ChatGPT

Accelerate building the Bing ecosystem and enhance performance with plugins

Microsoft and OpenAI will support and grow the AI plugin ecosystem through interoperability. This means developers can now use a single platform to build and submit plugins for consumers and businesses, including ChatGPT, Bing, Dynamics 365 Copilot, Microsoft 365 Copilot, and Windows Copilot.

Bing is adding support for a variety of plugins as part of this shared plugin platform. In addition to the previously announced OpenTable and Wolfram Alpha plug-ins, Microsoft announced the addition of Expedia, Instacart, Kayak, Klarna, Redfin, TripAdvisor and Zillow to the Bing ecosystem.

With plug-ins built directly into the chat system, Bing makes relevant suggestions based on users' conversations. For example, users can use the OpenTable plug-in to ask about restaurants and other related topics. In addition, Microsoft also stated that such functions will be able to be used in the Bing application on the mobile terminal, which will greatly expand the application scenarios.

Additionally, dozens of businesses are becoming partners with the Bing Chat plugin:

Bing plugin

Meanwhile, developers can now use plug-ins to integrate their apps and services into Microsoft 365 Copilot. Plugins for Microsoft 365 Copilot include ChatGPT and Bing plugins, as well as Teams messaging extensions and the Power Platform connector.

Integrate Bing into Copilot

At the same time, Microsoft has integrated the power of Bing Chat into Windows 11 with the release of Windows Copilot. Working together, Windows Copilot and Bing Chat (including the plug-in platform shared with Bing and OpenAI) will enable these plug-ins to be enhanced through apps on Windows. It will make it easier than ever to get personalized answers, relevant recommendations and take quick action.

In addition to Windows, Microsoft has also natively integrated its universal plug-in platform into the Edge browser, making it the first browser to integrate AI search.

Dev Home: Improving Developer Productivity on Windows 11

We know that developers are constantly dealing with manual development setups at work, not only with frequent clicks, but also with multiple tool logins, sub-optimal file system navigation, and context switching. All of these can seriously affect their development efficiency.

Therefore, in order to improve the productivity of developers, Microsoft launched a new productivity partner Dev Home in Windows 11. A preview of Dev Home is available now, with WinGet configuration feature for easier and faster setup, Dev Drive feature for enhanced file system capabilities, and a new custom control panel for tracking all workflows and tasks in one place . These can simplify the workflow of developers.

Dev Home can be easily linked to GitHub to easily install the tools and packages you need.

Dev Home can also configure user coding environments on the cloud using Microsoft Dev Box and GitHub Codespaces. In conclusion, with Dev Home designed for developers, Microsoft is providing the ultimate productivity partner so they can focus on what they do best — writing code.

Dev home address: https://github.com/microsoft/DevHome

WinGet configures unattended and reliable development machine setup

With the new WinGet configuration, developers can be ready to code in just a few clicks. This unattended, reliable and repeatable mechanism allows developers to skip the manual work of setting up new machines or launching new projects, eliminating the hassle of downloading the correct versions of software, packages, tools and frameworks, and application settings. Setup time can be reduced from days to hours.

Dev Drive: New storage volumes tailored for developers

Users often encounter situations where they need to deal with repositories containing thousands of files and directories, which presents a challenge for I/O operations such as builds. Now, Microsoft has launched a new type of storage volume, Dev Drive, which combines performance and security, and is customized for developers.

Based on the Resilient File System, Dev Drive provides up to 30% file system improvement in build time for file I/O scenarios.

Dev Home makes it easy to set up a Dev Drive during environment setup. It's great for hosting project source code, working folders, and package caches. Microsoft will provide a preview of Dev Drive later this week.

Efficiently track user workflow in Dev Home

Dev Home also helps users manage running projects, efficiently track all coding tasks or requests made, and track CPU and GPU performance by adding GitHub widgets. Additionally, Microsoft is working with Team Xbox to integrate GDK into Dev Home, making it easier to start game creation.

Windows Terminal powered by GitHub Copilot X

Users of GitHub Copilot will be able to use natural language AI models to recommend commands, explain errors, and take actions within the Terminal application through the inline and experimental chat experiences.

In addition, Microsoft is also experimenting with AI features supported by GitHub Copilot in developer tools such as WinDBG to help users improve productivity. The GitHub Copilot Chat waitlist is open now, and Microsoft will enable access to these features soon.

Make every Windows 11 developer an AI developer

In addition, Microsoft continues to invest in important tools to democratize the building of applications for the new era of artificial intelligence. Regardless of whether users are developing on x86/x64 or Arm64, Microsoft hopes that users can easily bring AI-driven experiences to Windows applications in cloud computing and edge computing.

At Build last year, Microsoft announced a new development model, Hybrid Loop, that enables hybrid AI scenarios across Azure and client devices. Similarly, on today's Build, Microsoft said its vision has been realized, using ONNX Runtime as a gateway for Windows AI and Olive, the tool chain created by Microsoft relieves users of the burden of optimizing various Windows and other device models. With ONNX Runtime, third-party developers can access tools Microsoft uses internally to run AI models on Windows or other devices across CPUs, GPUs, NPUs, or hybrids with Azure.

When running a model, ONNX Runtime now supports running the same API on-device or in the cloud, supporting hybrid inference scenarios, in addition, user applications can use local resources and switch to the cloud when needed. With the new Azure EP preview, users can connect to models deployed in AzureML and even connect to Azure OpenAI services. With just a few lines of code, users can specify cloud endpoints and define criteria for when to use the cloud. As a result, users can better control costs because Azure EP allows users the flexibility to choose to use a larger model in the cloud or use a smaller local model at runtime.

Users can also optimize models for different hardware using Olive, an extensible toolchain that combines cutting-edge technologies for model compression, optimization, and compilation. Based on this, users can use ONNX Runtime across platforms such as Windows, iOS, Android and Linux.

In summary, both ONNX Runtime and Olive help to speed up the deployment of AI models into applications. ONNX Runtime makes it easier for users to create stunning AI experiences on Windows and other platforms while reducing engineering effort and increasing performance.

New features and experiences in the Microsoft Store

Microsoft released a number of new features for the Microsoft Store, related to AI technology, including:

• Microsoft Store AI Hub: This is a new feature that will be launched soon, and the developer community and Microsoft will jointly build the best AI experience;
• AI-generated keywords: Microsoft has added a new tool to Partner Center that uses AI to generate keywords and search tags for apps in the Microsoft Store, improving the discoverability of those apps;
• AI-generated review summaries: Microsoft leveraged AI technology to compile thousands of reviews into one simple summary, making it faster and easier for customers to understand user reviews and feedback for their apps.

The above is just the content in the keynote on the first day of the multi-day build developer conference, and more content will be released in the future.

From this, we can see that Microsoft teamed up with OpenAI to create an AI universe, which not only transforms and upgrades its own products, but also radiates and affects the entire technology community.

Perhaps, this wave of AI has just begun.

Reference link:

https://blogs.windows.com/windowsdeveloper/2023/05/23/bringing-the-power-of-ai-to-windows-11-unlocking-a-new-era-of-productivity-for-customers- and-developers-with-windows-copilot-and-dev-home/