Text generation API reference

`generation.generate()`

Generates text from a prompt. Runs remotely when API or JWT credentials are configured. Runs locally when mx.set_execution_mode("local") is set, or in auto mode when a local model is loaded and no credentials are configured.

Core Parameters

string

required

The input text to generate a continuation for.

integer

default:"128"

Maximum number of tokens to generate.

string

default:"top-k"

Token sampling strategy. See Sampling Methods below.

float

default:"0.7"

Controls randomness. Lower values (0.1-0.5) produce more focused output; higher values (0.8-1.2) increase creativity.

Sampling Parameters

integer

default:"50"

For top-k sampling: number of top tokens to sample from.

float

default:"0.9"

For top-p sampling: cumulative probability threshold (0.0-1.0).

float

For min-p sampling: minimum relative probability threshold.

float

For typical sampling: typical probability threshold.

integer

For ads sampling: number of candidate tokens per step (2-10).

float

For ads sampling: quality vs diversity balance (0.1-0.5).

Steering Parameters

string | dict

A steering vector ID string (remote) or a {layer_index: tensor} dict (local).

float

default:"0"

Multiplier for the steering vector magnitude.

string

A named preset (e.g., "brevity", "truthfulness"). Applied as a pre-configured steering configuration.

Constrained Generation

dict

JSON schema to constrain output format. Used with guided-generation sampling.

string

Regex pattern to constrain output. Used with guided-generation sampling.

string

Grammar specification to constrain output. Used with guided-generation sampling.

Advanced Parameters

string

Model name for speculative decoding (ssd sampling method).

list[str]

List of model names for ensemble-sampling.

integer

default:"1"

Number of candidates to generate and score when using the policy-backed path.

boolean

default:"false"

Enables adaptive temperature during retries.

list[float]

Temperature values to use across retry rounds. Setting this also enables adaptive temperature.

boolean

default:"false"

Enables adaptive nucleus sampling during retries.

list[float]

Top-p values to use across retry rounds. Setting this also enables adaptive top-p.

boolean

default:"false"

Regenerates when the selected candidate’s confidence falls below confidence_threshold.

float

default:"0.5"

Minimum candidate confidence when confidence-triggered regeneration is enabled.

list[str]

Python unit-test snippets used by the policy verifier for generated code.

dict

Inline policy configuration. See Policies.

string

ID of a saved policy to apply during generation.

boolean

default:"false"

Requests trace information from policy execution. generation.generate() still returns the output string.

Returns: A plain string containing the generated text.

import mechanex as mx

mx.set_key("ax_your_key_here")
output = mx.generation.generate(
    "Summarize the concept of contrastive activation addition in two sentences.",
    max_tokens=128,
    sampling_method="top-p",
    top_p=0.9,
    temperature=0.7,
)
print(output)

Sampling Methods

Method	Description	Key Parameters
`greedy`	Deterministic; always picks the highest-probability token. Best for factual or structured outputs.	—
`top-k`	Samples from the top K tokens by probability. Default K is 50.	`top_k`
`top-p`	Nucleus sampling; samples from tokens covering the probability threshold. Balanced and natural.	`top_p`
`min-p`	Filters tokens below a relative probability threshold. Adapts dynamically to model confidence.	`min_p`
`typical`	Selects tokens based on local entropy, favoring “typical” continuations.	`typical_p`
`ads`	Adaptive Determinantal Sampling; maximizes diversity. Remote-only.	`ads_subset_size`, `ads_beta`
`guided-generation`	Constrains output to match a JSON schema, regex, or grammar.	`json_schema`, `regex_pattern`, `grammar`
`constrained-beam-search`	Generates multiple candidates and selects the best.	—
`ssd` / `speculative-decoding`	Uses a smaller draft model for faster generation.	`draft_model`
`ensemble-sampling`	Combines outputs from multiple models via voting.	`ensemble_models`

Applying a Steering Vector

Pass a vector ID (returned from steering.generate_vectors() or created in Spectra) or a local {layer: tensor} dict:

output = mx.generation.generate(
    "Tell me about this situation.",
    steering_vector="sv_abc123",
    steering_strength=1.5,
)

For local generation with a custom vector:

import torch

my_vector = {11: torch.randn(1, 768)}  # layer 11, hidden dim 768
output = mx.generation.generate(
    "Tell me about this situation.",
    steering_vector=my_vector,
    steering_strength=0.8,
)

Local vs Remote

Remote failures are surfaced when auto mode resolves to remote. Call mx.set_execution_mode("local") to force a loaded local model. Plain local generation supports greedy, top-k, top-p, min-p, and typical. Policy-backed local generation can also run saved policies, constraints, retries, and verifiers. ADS and steering perceptrons are not supported locally.

Getting Started

SDK Reference

CLI

Text generation API reference

`generation.generate()`

Core Parameters

Sampling Parameters

Steering Parameters

Constrained Generation

Advanced Parameters

Sampling Methods

Applying a Steering Vector

Local vs Remote

​generation.generate()

​Core Parameters

​Sampling Parameters

​Steering Parameters

​Constrained Generation

​Advanced Parameters

​Sampling Methods

​Applying a Steering Vector

​Local vs Remote

`generation.generate()`

Core Parameters

Sampling Parameters

Steering Parameters

Constrained Generation

Advanced Parameters

Sampling Methods

Applying a Steering Vector

Local vs Remote