Skip to main content

​
Overview

mx.serve() starts a local FastAPI server that exposes OpenAI-compatible chat and completion endpoints. This makes it straightforward to use Axionic-hosted or locally loaded models with any tool or library that speaks the OpenAI API format, without changing your existing code.

​
serve()

​
model
string
The model name to serve. Uses the currently loaded local model if omitted, or the default remote model.
​
host
string
default:"0.0.0.0"
Host address to bind the server to.
​
port
integer
default:"8000"
Port to listen on.
​
use_vllm
boolean
default:"false"
If true, uses vLLM as the inference backend instead of the default PyTorch / remote path.
​
corrected_behaviors
list[str]
Behavior names to monitor and auto-correct for all requests handled by this server instance.

​
Endpoints

Once the server is running:
EndpointMethodDescription
/v1/chat/completionsPOSTOpenAI chat format (messages array)
/v1/completionsPOSTOpenAI completion format (prompt string)
Both accept standard OpenAI request bodies.

​
Axionic Extensions

Pass Axionic-specific parameters via extra_body on the OpenAI Python client, or include them directly in the JSON request body:
FieldTypeDescription
steering_vector_idstringID of a steering vector to apply during generation
steering_strengthfloatMultiplier for the steering vector magnitude
policyobjectInline runtime policy to apply during generation
policy_idstringSaved policy ID to apply during generation
behavior_nameslist[str]Behaviors to monitor; correction applied if drift detected
force_steeringlist[str]Behaviors to steer toward unconditionally
mx.serve(corrected_behaviors=[...]) applies those behavior monitors to every request handled by that server instance. The CLI form, mechanex serve, exposes --host, --port, and --use-vllm; use Python when you need process-wide corrected_behaviors.

​
Example

import threading
import mechanex as mx
from openai import OpenAI

mx.set_key("ax_your_key_here")

# Start the server in a background thread
server_thread = threading.Thread(
    target=mx.serve,
    kwargs={"port": 8000, "corrected_behaviors": ["safety"]},
    daemon=True,
)
server_thread.start()

# Use the standard OpenAI client pointed at the local server
client = OpenAI(
    api_key="ax_your_key_here",
    base_url="http://localhost:8000/v1",
)

response = client.chat.completions.create(
    model="mechanex-mini",
    messages=[{"role": "user", "content": "Explain gradient descent."}],
    extra_body={
        "steering_vector_id": "sv_abc123",
        "steering_strength": 1.2,
    },
)
print(response.choices[0].message.content)