Microsoft unveils Windows AI Foundry for AI-powered PC apps

Microsoft is replacing 'Copilot Runtime' with Windows AI Foundry to help developers build, experiment, and reach users with AI experiences in their apps.

Windows AI Foundry will not only include Microsoft's model but other open-source models as well, including those from Nvidia, such as NVIDIA NIM.

With these models, developers can integrate new AI experiences and models in their applications.

"In addition, Windows AI Foundry offers ready-to-use AI APIs that are powered by Windows inbox models on Copilot+ PCs for key language and vision tasks, like text intelligence, image description, text recognition, custom prompt and object erase," Microsoft announced in a blog post.

"We are announcing new capabilities like LoRA (low-rank-adaption) for finetuning our inbox SLM, Phi Silica, with custom data. We are also announcing new APIs for semantic search and knowledge retrieval so developers can build natural language search and RAG (retrieval-augmented generation) scenarios in their apps with their custom data."

AI Foundry helps dev build new AI models for their PCs

AI Foundry in Windows also makes it easier for developers to use powerful AI models on their PCs without needing a lot of setup or optimization. It checks what kind of hardware your computer has and then shows which AI models will work best.

For example, AI Foundry automatically matches models to your device's hardware, so if you're a developer, you don't have to figure it out.

Here is the complete list of features:

• Ready-to-use models: It gives developers access to pre-trained open-source AI models (like those from Ollama, NVIDIA, and others) that are already optimized to run smoothly on different computer hardware, whether that's a CPU, GPU, or NPU (Neural Processing Unit).
• Foundry Local: This helps find and use the right models for your PC. You can install it using a simple command (winget install Microsoft.FoundryLocal).
• Developer tools: Developers also get a command-line tool (CLI) and an SDK (software kit) to easily test and add these models to their apps.

All of these features will soon be built directly into Windows 11 and the Windows App SDK.