2. GitHub Models Connection

• 2. GitHub Models Connection
  • Learning Goals
  • Introduction to GitHub Models
    • Highlights
    • API Endpoint
  • Step 1 - Generate a GitHub Access Token
    • Instructions
  • Step 2 - Store the Token Securely
  • Step 3 - Create a Test Script
  • Step 4 - Execute the Script
    • Example Output
  • Code Walkthrough
    • Creating the Client
      • What this does
    • Sending a Chat Request
    • Reading the Response
  • Recommended Models
  • Suggested Project Layout
  • Completion Checklist
  • Next

In this module, you’ll set up access to GitHub Models and make your first AI API request using Python.

Learning Goals

After completing this section, you should be able to:

• Understand what GitHub Models is
• Generate a GitHub Personal Access Token (PAT)
• Configure environment variables securely
• Connect to an AI model using the OpenAI SDK
• Send and receive responses from a model

Introduction to GitHub Models

GitHub provides a hosted AI inference service called GitHub Models, which gives developers access to several popular models through an OpenAI-compatible API.

Highlights

API Endpoint

https://models.github.ai/inference

Because the API follows the OpenAI format, you can use the standard OpenAI Python library without major changes.

Step 1 - Generate a GitHub Access Token

To use GitHub Models, you’ll need a Personal Access Token.

Instructions

1. Open GitHub token settings: https://github.com/settings/tokens
2. Select:

Generate new token -> Generate new token (classic)

1. Configure the token:

1. Click Generate token
2. Copy the token immediately and store it safely

[!IMPORTANT] GitHub only shows the token once.

Step 2 - Store the Token Securely

Create or update a .env file in your project root.

GITHUB_TOKEN=your_token_here
GITHUB_MODEL=gpt-4o-mini

This keeps secrets outside your source code.

Step 3 - Create a Test Script

Create a new Python file inside the current project root (lab/) for testing GitHub Models.

touch test_github_models_connection.py

Now add the following Python code.

"""
Module 2 - Test GitHub Models Connection

Run:
    python test_github_models_connection.py
    or
    uv run test_github_models_connection.py
"""

import asyncio
import os

from dotenv import load_dotenv
from openai import AsyncOpenAI

load_dotenv()


async def main():
    """Send a simple request to GitHub Models."""

    print("🔌 Connecting to GitHub Models...")

    client = AsyncOpenAI(
        api_key=os.getenv("GITHUB_TOKEN"),
        base_url="https://models.github.ai/inference",
    )

    print("📨 Sending request...")

    response = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "user",
                "content": "Reply with: Hello from GitHub Models!",
            }
        ],
    )

    message = response.choices[0].message.content

    print("\n💬 Model Response:", message)


if __name__ == "__main__":
    asyncio.run(main())

Step 4 - Execute the Script

Run the test file:

python test_github_models_connection.py
# or
uv run test_github_models_connection.py

Example Output

🔌 Connecting to GitHub Models...
📨 Sending request...

💬 Model Response: Hello from GitHub Models!

If you receive a valid response, your setup is working correctly.

Code Walkthrough

Creating the Client

client = AsyncOpenAI(
    api_key=os.getenv("GITHUB_TOKEN"),
    base_url="https://models.github.ai/inference",
)

What this does

Sending a Chat Request

response = await client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {
            "role": "user",
            "content": "Reply with: Hello from GitHub Models!",
        }
    ],
)

This sends a standard OpenAI-style chat request.

Reading the Response

message = response.choices[0].message.content

The generated text is stored inside the response object.

Recommended Models

For most workshop examples, gpt-4o-mini is a good default because it is fast and cost-efficient.

Suggested Project Layout

├── lab
│   ├── main.py
│   ├── pyproject.toml
│   ├── README.md
│   ├── requirements.txt
│   ├── test_github_models_connection.py
│   └── uv.lock

Completion Checklist

Next

Continue to 3. Microsoft Agent Framework Agents.