Chatbot Playground App with AKS KAITO and MS Foundry Hosted LLMs YouTube video by Roy Kim

My demo of my Streamlit chatbot playground for testing multiple AI providers and deployed LLMs from one app.

Switch between chat agents and compare AI backends, including MSFT Foundry and AKS-hosted KAITO models. Test to compare model behaviour with your prompts.

youtu.be/Od3lNH2_9ak?...

I tried out the new AKS Desktop app found at github.com/Azure/aks-de...

This is definitely a tool I use to help monitor and manage aks clusters on a regular basis.
The UI is very interactive and responsive. I can only imagine this desktop app will only get better from here.

#MVPSummit 2026 starts today. Can't wait to attend sessions and learn about product advancements related to #auzre #aks #AI #MicrosoftFoundry.

AI Document Ingestion and Querying with KAITO RAG Engine on Azure Kubernetes YouTube video by Roy Kim

My New Video: Showing capabilities of my RAG app on #Kubernetes using the Kubernetes AI Toolchain Operator on AKS.

• Ingest documents into a vector store
• Document ingestion and prompt management
• Running inference with the Phi-4 LLM hosted directly on AKS

youtu.be/2AP1LIeK45g

#AI #RAG #AKS

Kicking off the day at #agentcon Toronto. Excited to get plugged in with the local community sharing knowledge and experience with building AI agents with Microsoft platform. #genai #mvpbuzz #microsoft

Overview of Hosting LLMs and RAG Engine on Azure Kubernetes with KAITO, Tue, Mar 17, 2026, 6:00 PM | Meetup KAITO RAG Engine on Azure Kubernetes Service is a Kubernetes-native way to run a full Retrieval-Augmented Generation (RAG) backend fully hosted inside your own Azure enviro

Sign up for my tech talk: www.meetup.com/azureqc/even...

Doc Ingestion CLI for AKS KAITO RAG Engine I built a ragengine-ingest-docs CLI tool simplifies managing documents in the AKS KAITO Retrieval Augmented Generation Engine. It supports creating, updating, listing, and querying documents, enhancing demos and workshops by clearly demonstrating document indexing and retrieval behaviors.

Doc Ingestion CLI for AKS KAITO RAG Engine

I built a ragengine-ingest-docs CLI tool simplifies managing documents in the AKS KAITO Retrieval Augmented Generation Engine. It supports creating, updating, listing, and querying documents, enhancing demos and workshops by clearly demonstrating document…

aks-demos/aks-kaito at main · RoyKimYYZ/aks-demos Azure Kubernetes Demos for the purposes of sharing my knowledge to the technology community - RoyKimYYZ/aks-demos

Want to run LLMs on AKS without the infrastructure headache? 🚀
Check out my repo: 🔗
github.com/RoyKimYYZ/ak...
✅ Automated GPU provisioning
✅ One-click OSS model deployment (Phi, Llama, Falcon)
✅ Private & secure inference endpoints
#AKS #Kubernetes #Azure #GenerativeAI #LLMOps #KAITO #mvpbuzz

GitHub - RoyKimYYZ/azure-ai-apps: Azure AI Apps - RAG, Agents, and Chat applications using Azure AI Foundry, Azure Kubernetes Service and other Azure AI related services. Azure AI Apps - RAG, Agents, and Chat applications using Azure AI Foundry, Azure Kubernetes Service and other Azure AI related services. - RoyKimYYZ/azure-ai-apps

I like to share my github repo of my collection of demo AI apps built for Azure. I share my learning journey of building chatbot demos, RAG demos against Azure Kubernetes and AI Foundry.
Please click Watch and STAR as I continue to add and develop demos.
github.com/RoyKimYYZ/az...

Autoscale KAITO inference workloads on AKS using KEDA | AKS Engineering Blog Learn how to autoscale KAITO inference workloads on AKS with KEDA to handle varying requests and optimize GPU utilization for AI models at scale.

Autoscaling LLM inference on AKS

KAITO enables native, GPU-aware autoscaling for inference workloads without reinventing the wheel.
blog.aks.azure.com/2026/02/03/a...
#Azure #AKS #GenAI #LLMs

Installing KAITO v0.8.x RAG Engine on Azure Kubernetes Service: Architecture and Chat Demo YouTube video by Roy Kim

My Video:
Explore KAITO RAG Engine on Azure Kubernetes: Architecture and Chatbot Demo.

KAITO RAG Engine on Azure Kubernetes Service is a Kubernetes-native way to run a full Retrieval-Augmented Generation backend fully hosted inside your own Azure environment. #azure #RAG #AI
youtu.be/JLY6IFJIK7M

Haniel Croitoru and I hosted another Toronto Area MVP Dinner.
We discussed collaborations of contributing to our technical communities of Microsoft Technologies such as Power Platform, Business Apps, Excel, Azure and of course AI.
Watch out for content and support your favourite Microsoft MVPs.

Using Streamlit Chatbot UI with AKS KAITO Language Model Inferences YouTube video by Roy Kim

My video: youtube.com/watch?v=U1gh...
I walk through deploying a Streamlit chat app that talks to a KAITO-hosted LLM on AKS, including ingress routing for rapid testing and demos.

Using Streamlit Chatbot UI with AKS KAITO Language Model Inferences This blog post discusses setting up a chatbot UI using Streamlit alongside a deployed language model inference service in Azure Kubernetes. It details the process of testing the inference service w…

My blog post:
How to install a chatbot UI with Streamlit that talks to a KAITO language model inference service running on Azure Kubernetes Service (AKS)!
Instead of using curl, you get a browser-based chat UI hooked into your own language model endpoint.
tinyurl.com/ywr53vpe
#Azure #AKS #KAITO #AI

Install KAITO v0.8.x on Azure Kubernetes Service With Phi-4 Language Model The Kubernetes AI Toolchain Operator (KAITO) automates AI/ML model processes in Kubernetes, enhancing control and security. This setup involves creating an AKS cluster, installing KAITO, and config…

My Blog Post: roykim.ca/2025/12/21/i...
I will show my installation steps that are based on with KAITO release v0.8.x, yet I have added ingress controller installation for ease of testing phi-4 model. #azure #ai #kubernetes #aks

In Toronto, we caught up with fellow Microsoft MVPs. We reconnect, share experiences, delve into the latest tech trends, and brainstorm ways to support the technical community dedicated to advancing Microsoft Technologies.
#MicrosoftMVPs #TechCommunity #mvpbuzz

2025 Microsoft Most Valuable Professional (MVP) was issued by Microsoft MVP and Student Ambassadors Communities to Roy Kim. The Microsoft MVP Program recognizes outstanding members of technical communities for their community participation and willingness to help others. Above all else, it is a people-powered program, made...

View my verified achievement from Microsoft MVP Communities. www.credly.com/badges/d0761... via @credly

Microsoft | Microsoft Cloud and AI Bootcamp Join Microsoft Cloud and AI Bootcamp, a digital experience designed to accelerate your AI ambitions, from strategy to secure, scalable execution.

Excited to share an opportunity to register for the AI Bootcamp hosted by Microsoft! Join us on August 19-20, 2025, to gain valuable insights from real-world success stories, expert-led demos, and partner solutions showcasing AI in action. Register now:
www.microsoft.com/en-us/AIBoot...
#AI

Today, I was awarded once again for a 9th year in the Microsoft MVP Award program. I am awarded in the Azure category. Being an MVP is about learning, sharing and building community.

In a recent AKS Community Call, I was featured for my blog post series on Setting LLM into an AKS cluster with KAITO -https://tinyurl.com/3js5wkt7

Feel free to follow and register for future AKS Community calls by the Microsoft AKS product team - blog.aks.azure.com/webinars/

Well Architected Framework With The Azure Verified Module For Azure Kubernetes I came across the Azure Verified Module for Azure Kubernetes Service and in its Github repo I found a Well Architected Framework (WAF) Aligned example for deploying this Terraform module. So asked …

My new Blog Post:
Do you wonder how the Azure Verified Module in Terraform for AKS is aligned with the Azure Well Architected Framework? I explain with its WAF aligned terraform example.
roykim.ca/2025/03/10/w...

Exploring Azure Verified Module for Azure Kubernetes Service I have been testing out the Azure Verified Module for Azure Kubernetes Service that can be found in the Terraform Registry at This module came out in October 2024 so its fairly new. This module is …

My Blog Post:
The Azure Verified Module for #Azure Kubernetes Service offers enterprise-grade features, Microsoft support, and best practices for production environments. While simpler alternatives exist, AVM provides better abstraction and compliance.
roykim.ca/2025/03/16/e... #terraform #aks

At the AI Global Bootcamp 2025, I presented Fine-tuning a language model on Azure Kubernetes with KAITO that enables you to fine-tune relatively easily with NVIDIA GPUs. To read about my experience with fine-tuning, visit my blog www.roykim.ca #azure #ai #k8s #mvpbuzz

Fine Tuning the Phi3 Language Model Using Azure Kubernetes Service YouTube video by CTTDNUG

Check out my Livestream on Fine-Tuning a language model on Azure Kubernetes Service
www.youtube.com/watch?v=zK3c...

Azure Verified Module Storage Account: Firewalls and Virtual Network Configuration I have been trying out the Azure Verified Module avm-res-storage-storageaccount. module’s repo: Hashicorp Terraform Registry: I like show some simple networking scenarios in the Firewalls and…

My blog post: I have been playing with #Azure Verified Modules for the Storage Account. I like to explain the network configuration and hope it helps. #AVM #terraform #mvpbuzz

roykim.ca/2025/02/17/a...

Permissions with Azure AI Foundry: Safety And Security As I was starting to try out Azure Foundry Safety and Security feature, I confronted with the error “Your account does not have access to this resource, please contact your resource owner to …

My Blog Post: roykim.ca/2025/02/12/p...
While using Azure Foundry Safety and Security, I encountered access issues despite having owner permissions. Proper role assignments are crucial for accessing underlying Azure AI services.

An Intro For Using Azure Verified Modules With Terraform Azure Verified Modules (AVM) enhance Azure deployment by providing officially supported, consistent Terraform modules from Microsoft, improving reliability and user experience compared to previous …

My blog post:
roykim.ca/2025/02/12/a...
Azure Verified Modules (AVM) enhance #Azure deployment by providing officially supported, consistent #Terraform modules from Microsoft, improving reliability and user experience compared to previous azurerm modules.

Azure Terraform Starter Project Example This project is sample code for those who are beginners to Terraform for Azure and as a quick reference. There are many ways to structure a terraform project in your repo, but I just want to show a…

My blog post:
This project demonstrates a simple #Terraform setup for #Azure beginners, focusing on deploying a storage account and blob container with organized code structure and reusable modules.
roykim.ca/2025/02/06/a...

Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 1: Intro I uncover the process and workings of fine-tuning a large language model on a dataset. Scenario for fine-tuning a model can be on corporate data such as a knowledge base, product information, human…

Ever wonder what the process of fine-tuning is like? Check out my new blog series:
Fine-Tuning Phi-3 Language model using #KAITO on #Azure #Kubernetes
shorturl.at/IAHcX
shorturl.at/YQJyK
shorturl.at/90Zzd
shorturl.at/IFqpD

#AI #microsoft #genAI #artificialintelligence

Deep Dive Into Fine-Tuning An LM Using KAITO on AKS – Part 1: Intro I uncover the process and workings of fine-tuning a large language model on a dataset. Scenario for fine-tuning a model can be on corporate data such as a knowledge base, product information, human…

My Post: roykim.ca/2025/01/11/d...
Exploring fine-tuning large language models using specific datasets, particularly focusing on corporate data. It outlines the steps of selecting pre-trained models, preparing datasets, training, and evaluating. #azure #mvpbuzz #ai #fine-tuning #kubernetes #aks #llm