Skip to main content

​
generation.generate()

Generates text from a prompt. Runs remotely when API or JWT credentials are configured. Runs locally when mx.set_execution_mode("local") is set, or in auto mode when a local model is loaded and no credentials are configured.

​
Core Parameters

​
prompt
string
required
The input text to generate a continuation for.
​
max_tokens
integer
default:"128"
Maximum number of tokens to generate.
​
sampling_method
string
default:"top-k"
Token sampling strategy. See Sampling Methods below.
​
temperature
float
default:"0.7"
Controls randomness. Lower values (0.1-0.5) produce more focused output; higher values (0.8-1.2) increase creativity.

​
Sampling Parameters

​
top_k
integer
default:"50"
For top-k sampling: number of top tokens to sample from.
​
top_p
float
default:"0.9"
For top-p sampling: cumulative probability threshold (0.0-1.0).
​
min_p
float
For min-p sampling: minimum relative probability threshold.
​
typical_p
float
For typical sampling: typical probability threshold.
​
ads_subset_size
integer
For ads sampling: number of candidate tokens per step (2-10).
​
ads_beta
float
For ads sampling: quality vs diversity balance (0.1-0.5).

​
Steering Parameters

​
steering_vector
string | dict
A steering vector ID string (remote) or a {layer_index: tensor} dict (local).
​
steering_strength
float
default:"0"
Multiplier for the steering vector magnitude.
​
steering_preset
string
A named preset (e.g., "brevity", "truthfulness"). Applied as a pre-configured steering configuration.

​
Constrained Generation

​
json_schema
dict
JSON schema to constrain output format. Used with guided-generation sampling.
​
regex_pattern
string
Regex pattern to constrain output. Used with guided-generation sampling.
​
grammar
string
Grammar specification to constrain output. Used with guided-generation sampling.

​
Advanced Parameters

​
draft_model
string
Model name for speculative decoding (ssd sampling method).
​
ensemble_models
list[str]
List of model names for ensemble-sampling.
​
best_of_n
integer
default:"1"
Number of candidates to generate and score when using the policy-backed path.
​
adaptive_temperature
boolean
default:"false"
Enables adaptive temperature during retries.
​
adaptive_temperature_schedule
list[float]
Temperature values to use across retry rounds. Setting this also enables adaptive temperature.
​
adaptive_top_p
boolean
default:"false"
Enables adaptive nucleus sampling during retries.
​
adaptive_top_p_schedule
list[float]
Top-p values to use across retry rounds. Setting this also enables adaptive top-p.
​
confidence_triggered_regeneration
boolean
default:"false"
Regenerates when the selected candidate’s confidence falls below confidence_threshold.
​
confidence_threshold
float
default:"0.5"
Minimum candidate confidence when confidence-triggered regeneration is enabled.
​
code_unit_tests
list[str]
Python unit-test snippets used by the policy verifier for generated code.
​
policy
dict
Inline policy configuration. See Policies.
​
policy_id
string
ID of a saved policy to apply during generation.
​
include_trace
boolean
default:"false"
Requests trace information from policy execution. generation.generate() still returns the output string.
Returns: A plain string containing the generated text. 
import mechanex as mx

mx.set_key("ax_your_key_here")
output = mx.generation.generate(
    "Summarize the concept of contrastive activation addition in two sentences.",
    max_tokens=128,
    sampling_method="top-p",
    top_p=0.9,
    temperature=0.7,
)
print(output)

​
Sampling Methods

MethodDescriptionKey Parameters
greedyDeterministic; always picks the highest-probability token. Best for factual or structured outputs.—
top-kSamples from the top K tokens by probability. Default K is 50.top_k
top-pNucleus sampling; samples from tokens covering the probability threshold. Balanced and natural.top_p
min-pFilters tokens below a relative probability threshold. Adapts dynamically to model confidence.min_p
typicalSelects tokens based on local entropy, favoring “typical” continuations.typical_p
adsAdaptive Determinantal Sampling; maximizes diversity. Remote-only.ads_subset_size, ads_beta
guided-generationConstrains output to match a JSON schema, regex, or grammar.json_schema, regex_pattern, grammar
constrained-beam-searchGenerates multiple candidates and selects the best.—
ssd / speculative-decodingUses a smaller draft model for faster generation.draft_model
ensemble-samplingCombines outputs from multiple models via voting.ensemble_models

​
Applying a Steering Vector

Pass a vector ID (returned from steering.generate_vectors() or created in Spectra) or a local {layer: tensor} dict: 
output = mx.generation.generate(
    "Tell me about this situation.",
    steering_vector="sv_abc123",
    steering_strength=1.5,
)
For local generation with a custom vector: 
import torch

my_vector = {11: torch.randn(1, 768)}  # layer 11, hidden dim 768
output = mx.generation.generate(
    "Tell me about this situation.",
    steering_vector=my_vector,
    steering_strength=0.8,
)

​
Local vs Remote

Remote failures are surfaced when auto mode resolves to remote. Call mx.set_execution_mode("local") to force a loaded local model. Plain local generation supports greedy, top-k, top-p, min-p, and typical. Policy-backed local generation can also run saved policies, constraints, retries, and verifiers. ADS and steering perceptrons are not supported locally.