Inference
Mixtral-8x7B Inference with SageMaker
Enter a prompt below to get a response from the model.
Runtime request path
Next.js
Route Handler
API Gateway
HTTPS
Lambda
Invoke
SageMaker
Endpoint
Mixtral
8x7B + adapter
Response
Normalized
Enter your prompt
Model response
No output yet
Generation parameters
Fixed by the deployed Lambda
The handler sets these values and applies them to every request.Also set server-side: do_sample true, top_k 50, return_full_text false, stop sequence </s>.