(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [api_utils.py:347] 
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [api_utils.py:347]        █     █     █▄   ▄█
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [api_utils.py:347]  ▄▄ ▄█ █     █     █ ▀▄▀ █  version 0.29.0
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [api_utils.py:347]   █▄█▀ █     █     █     █  model   /workshop/hf-cache/hub/models--openjev--openjev-FP8/snapshots/4ec320f267401e67c9be04d5df1be4d2b6b64f10
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [api_utils.py:347]    ▀▀  ▀▀▀▀▀ ▀▀▀▀▀ ▀     ▀
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [api_utils.py:347] 
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [api_utils.py:286] non-default args: {'model_tag': '/workshop/hf-cache/hub/models--openjev--openjev-FP8/snapshots/4ec320f267401e67c9be04d5df1be4d2b6b64f10', 'host': '127.0.0.1', 'model': '/workshop/hf-cache/hub/models--openjev--openjev-FP8/snapshots/4ec320f267401e67c9be04d5df1be4d2b6b64f10', 'trust_remote_code': True, 'max_model_len': 16384, 'quantization': 'fp8', 'max_logprobs': 64, 'served_model_name': ['openjev-fp8-largecard'], 'gpu_memory_utilization': 0.9, 'enable_prefix_caching': True, 'limit_mm_per_prompt': {'image': 0}, 'max_num_seqs': 1, 'gdn_prefill_backend': 'triton'}
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [model.py:684] Resolved architecture: Qwen3_5ForConditionalGeneration
(APIServer pid=331487) INFO 2026-09-22T04:09:23Z [model.py:2021] Using max model len 16384
(APIServer pid=331487) INFO 2026-09-22T04:09:26Z [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=8192.
(APIServer pid=331487) INFO 2026-09-22T04:09:26Z [config.py:625] Mamba cache mode is set to 'align' for Qwen3_5ForConditionalGeneration by default when prefix caching is enabled
(APIServer pid=331487) WARNING 2026-09-22T04:09:26Z [vllm.py:1211] Auto-disabled DeepGemm for model_type=qwen3_5_text on Blackwell. DeepGemm E8M0 scale format causes accuracy degradation for this architecture. Falling back to CUTLASS. To disable DeepGemm globally, set VLLM_USE_DEEP_GEMM=0.
(APIServer pid=331487) INFO 2026-09-22T04:09:26Z [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=331487) INFO 2026-09-22T04:09:26Z [compilation.py:329] Enabled custom fusions: norm_quant, act_quant
(APIServer pid=331487) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
(EngineCore pid=332027) INFO 2026-09-22T04:09:45Z [core.py:123] Initializing a V1 LLM engine (v0.29.0) with config: model='/workshop/hf-cache/hub/models--openjev--openjev-FP8/snapshots/4ec320f267401e67c9be04d5df1be4d2b6b64f10', speculative_config=None, tokenizer='/workshop/hf-cache/hub/models--openjev--openjev-FP8/snapshots/4ec320f267401e67c9be04d5df1be4d2b6b64f10', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=16384, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=fp8, quantization_config=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=openjev-fp8-largecard, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['+quant_fp8', 'none', '+quant_fp8'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_compute_ple_ngram_ids', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 2, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, enable_jit_warmup=True, enable_bf16x3_router_gemm=False, moe_backend='auto', linear_backend='auto')
(EngineCore pid=332027) INFO 2026-09-22T04:09:48Z [parallel_state.py:1775] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_db63527628864e0ebc4531fbbfd6283f backend=nccl
(EngineCore pid=332027) INFO 2026-09-22T04:09:48Z [parallel_state.py:2119] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A
(EngineCore pid=332027) INFO 2026-09-22T04:09:48Z [gpu_worker.py:429] Using V2 Model Runner
(EngineCore pid=332027) INFO 2026-09-22T04:09:48Z [model_runner.py:382] Loading model from scratch...
(EngineCore pid=332027) INFO 2026-09-22T04:09:49Z [cuda.py:551] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=332027) INFO 2026-09-22T04:09:49Z [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=332027) INFO 2026-09-22T04:09:49Z [__init__.py:695] Selected CutlassFp8BlockScaledMMKernel for Fp8LinearMethod
(EngineCore pid=332027) INFO 2026-09-22T04:09:49Z [qwen_gdn_linear_attn.py:167] Using Triton/FLA GDN prefill kernel (requested=triton, head_k_dim=128).
(EngineCore pid=332027) INFO 2026-09-22T04:09:49Z [qwen_gdn_linear_attn.py:519] GDN decode kernel: cuda
(EngineCore pid=332027) INFO 2026-09-22T04:09:49Z [cuda.py:492] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=332027) INFO 2026-09-22T04:09:49Z [flash_attn.py:897] Using FlashAttention version 2
(EngineCore pid=332027) INFO 2026-09-22T04:09:50Z [weight_utils.py:863] Filesystem type for checkpoints: EXT4. Checkpoint size: 28.30 GiB. Available RAM: 112.08 GiB.
(EngineCore pid=332027) INFO 2026-09-22T04:09:50Z [weight_utils.py:886] Auto-prefetch is disabled because the filesystem (EXT4) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:   0% Completed | 0/12 [00:00<?, ?it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:   8% Completed | 1/12 [00:04<00:45,  4.14s/it]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  17% Completed | 2/12 [00:05<00:25,  2.51s/it]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  25% Completed | 3/12 [00:06<00:14,  1.63s/it]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  33% Completed | 4/12 [00:06<00:10,  1.26s/it]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  42% Completed | 5/12 [00:07<00:07,  1.04s/it]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  50% Completed | 6/12 [00:08<00:05,  1.12it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  58% Completed | 7/12 [00:08<00:03,  1.26it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  67% Completed | 8/12 [00:09<00:03,  1.33it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  75% Completed | 9/12 [00:09<00:02,  1.41it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  83% Completed | 10/12 [00:10<00:01,  1.50it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards:  92% Completed | 11/12 [00:11<00:00,  1.50it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards: 100% Completed | 12/12 [00:11<00:00,  1.65it/s]
(EngineCore pid=332027) 
Loading safetensors checkpoint shards: 100% Completed | 12/12 [00:11<00:00,  1.03it/s]
(EngineCore pid=332027) 
(EngineCore pid=332027) INFO 2026-09-22T04:10:01Z [default_loader.py:430] Loading weights took 11.62 seconds
(EngineCore pid=332027) INFO 2026-09-22T04:10:02Z [model_runner.py:404] Model loading took 28.51 GiB memory and 13.973985 seconds
(EngineCore pid=332027) INFO 2026-09-22T04:10:02Z [topk_topp_sampler.py:46] FlashInfer top-p/top-k sampling disabled via VLLM_USE_FLASHINFER_SAMPLER=0.
(EngineCore pid=332027) INFO 2026-09-22T04:10:02Z [interface.py:918] Setting attention block size to 784 tokens to ensure that attention page size is >= mamba page size.
(EngineCore pid=332027) INFO 2026-09-22T04:10:02Z [interface.py:942] Padding mamba page size by 0.13% to ensure that mamba page size and attention page size are exactly equal.
(EngineCore pid=332027) INFO 2026-09-22T04:10:02Z [utils.py:306] Using LBNHC KV cache layout.
(EngineCore pid=332027) [transformers] The `use_fast` parameter is deprecated and will be removed in a future version. Use `backend="torchvision"` instead of `use_fast=True`, or `backend="pil"` instead of `use_fast=False`.
(EngineCore pid=332027) [transformers] Qwen3VL video processing does not apply the per-frame pixel cap the reference implementation (qwen-vl-utils) applies, so some videos cost far more tokens than they would there. In v5.22 the capped behavior will become the default and `cap_pixels_per_frame` will be removed. Pass `cap_pixels_per_frame=True` to adopt the reference behavior now, or `False` to keep the current behavior and silence this warning.
(EngineCore pid=332027) INFO 2026-09-22T04:10:11Z [encoder_runner.py:131] Encoder cache will be initialized with a budget of 12288 tokens, and profiled with 1 video items of the maximum feature size.
(EngineCore pid=332027) INFO 2026-09-22T04:10:32Z [backends.py:1094] Using cache directory: /workshop/.cache/vllm/torch_compile_cache/1f91413f9d/rank_0_0/backbone for vLLM's torch.compile
(EngineCore pid=332027) INFO 2026-09-22T04:10:32Z [backends.py:1155] Dynamo bytecode transform time: 18.51 s
(EngineCore pid=332027) INFO 2026-09-22T04:11:06Z [backends.py:393] Compiling a graph for compile range (1, 8192) takes 31.91 s
(EngineCore pid=332027) INFO 2026-09-22T04:11:16Z [backends.py:920] collected artifacts: 65 entries, 21 artifacts, 23738966 bytes total
(EngineCore pid=332027) INFO 2026-09-22T04:11:16Z [decorators.py:719] saved AOT compiled function to /workshop/.cache/vllm/torch_compile_cache/torch_aot_compile/68c5a6dc0f2c201d2b474cd87ca1e2da1096f91157ef233b9354c637956adf0c/rank_0_0/model
(EngineCore pid=332027) INFO 2026-09-22T04:11:16Z [monitor.py:53] torch.compile took 62.21 s in total
(EngineCore pid=332027) /workshop/bench-vllm-venv/lib/python3.12/site-packages/triton/language/core.py:2284: UserWarning: tl.make_block_ptr is deprecated. Use TensorDescriptor or tl.make_tensor_descriptor instead.
(EngineCore pid=332027)   warn("tl.make_block_ptr is deprecated. Use TensorDescriptor or tl.make_tensor_descriptor instead.")
(EngineCore pid=332027) INFO 2026-09-22T04:12:27Z [monitor.py:81] Initial profiling/warmup run took 70.37 s
(EngineCore pid=332027) 
Capturing CUDA graphs (PIECEWISE):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):  50%|█████     | 1/2 [00:01<00:01,  1.25s/it]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 2/2 [00:02<00:00,  1.24s/it]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 2/2 [00:02<00:00,  1.24s/it]
(EngineCore pid=332027) 
Capturing CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 12.58it/s]
(EngineCore pid=332027) INFO 2026-09-22T04:12:32Z [model_runner.py:960] Graph capturing finished in 3 secs, took 0.07 GiB
(EngineCore pid=332027) INFO 2026-09-22T04:12:33Z [gpu_worker.py:625] Available KV cache memory: 55.15 GiB
(EngineCore pid=332027) INFO 2026-09-22T04:12:33Z [gpu_worker.py:640] CUDA graph memory profiling is enabled (default since v0.21.0). The current --gpu-memory-utilization=0.9000 is equivalent to --gpu-memory-utilization=0.8992 without CUDA graph memory profiling. To maintain the same effective KV cache size as before, increase --gpu-memory-utilization to 0.9008. To disable, set VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=0.
(EngineCore pid=332027) INFO 2026-09-22T04:12:33Z [kv_cache_utils.py:2032] GPU KV cache size: 699,050 tokens, Maximum concurrency for 16,384 tokens per request: 42.67x
(EngineCore pid=332027) INFO 2026-09-22T04:12:33Z [kernel_warmup.py:124] JIT kernel warmup starting.
(EngineCore pid=332027) INFO 2026-09-22T04:12:33Z [kernel_warmup.py:134] JIT kernel warmup finished in 0.00s.
(EngineCore pid=332027) INFO 2026-09-22T04:12:33Z [qwen_triton_warmup.py:270] Warming up Qwen Triton kernels for model_type=qwen3_5_text.
(EngineCore pid=332027) INFO 2026-09-22T04:12:34Z [kernel_warmup.py:309] Using FlashInfer autotune cache file: /workshop/.cache/vllm/flashinfer_autotune_cache/0.6.18/120f/1fd9ddada29197096e86e8102fde69aa7acc27376807256b31e5c0b92ffc9ed1/autotune_configs.json
(EngineCore pid=332027) 2026-09-22T04:12:34.910Z - INFO - autotuner.py:972 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(EngineCore pid=332027) INFO 2026-09-22T04:12:34Z [kernel_warmup.py:266] Running FlashInfer autotune with 8192 tokens.
(EngineCore pid=332027) 2026-09-22T04:12:34.938Z - INFO - autotuner.py:995 - flashinfer.jit: [Autotuner]: Autotuning process ends
(EngineCore pid=332027) 2026-09-22T04:12:34.949Z - INFO - autotuner.py:2652 - flashinfer.jit: [Autotuner]: Saved 0 configs to /workshop/.cache/vllm/flashinfer_autotune_cache/0.6.18/120f/1fd9ddada29197096e86e8102fde69aa7acc27376807256b31e5c0b92ffc9ed1/autotune_configs.json (0 new, 0 from previous config)
(EngineCore pid=332027) 
Capturing CUDA graphs (PIECEWISE):   0%|          | 0/2 [00:00<?, ?it/s]
Capturing CUDA graphs (PIECEWISE):  50%|█████     | 1/2 [00:00<00:00,  5.49it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 2/2 [00:01<00:00,  1.38it/s]
Capturing CUDA graphs (PIECEWISE): 100%|██████████| 2/2 [00:01<00:00,  1.55it/s]
(EngineCore pid=332027) 
Capturing CUDA graphs (FULL):   0%|          | 0/1 [00:00<?, ?it/s]
Capturing CUDA graphs (FULL): 100%|██████████| 1/1 [00:00<00:00, 12.46it/s]
(EngineCore pid=332027) INFO 2026-09-22T04:12:37Z [model_runner.py:960] Graph capturing finished in 2 secs, took 0.05 GiB
(EngineCore pid=332027) INFO 2026-09-22T04:12:37Z [gpu_worker.py:797] CUDA graph pool memory: 0.05 GiB (actual), 0.07 GiB (estimated), difference: 0.02 GiB (35.7%).
(EngineCore pid=332027) INFO 2026-09-22T04:12:37Z [gpu_worker.py:860] Free memory on device (93.49/94.97 GiB) on startup. Desired GPU memory utilization is (0.9, 85.47 GiB). Actual usage is 28.84 GiB for consumed memory (weights + non-torch), 1.49 GiB for peak activation, and 0.05 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=58998983373` (54.95 GiB) to fit into requested memory, or `--kv-cache-memory=67601841664` (62.96 GiB) to fully utilize gpu memory. Current kv cache memory in use is 55.15 GiB.
(EngineCore pid=332027) INFO 2026-09-22T04:12:42Z [jit_monitor.py:85] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.
(EngineCore pid=332027) INFO 2026-09-22T04:12:42Z [torch_utils.py:277] Reducing Torch threads from 32 to 1 for serving. Set OMP_NUM_THREADS in the external environment to override.
(EngineCore pid=332027) INFO 2026-09-22T04:12:42Z [core.py:361] init engine (profile, create kv cache, warmup model) took 160.25 s (compilation: 62.21 s)
(EngineCore pid=332027) INFO 2026-09-22T04:12:42Z [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(EngineCore pid=332027) INFO 2026-09-22T04:12:42Z [compilation.py:329] Enabled custom fusions: norm_quant, act_quant
(APIServer pid=331487) INFO 2026-09-22T04:12:43Z [entry.py:135] Supported tasks: ['generate']
(APIServer pid=331487) INFO 2026-09-22T04:12:44Z [hf.py:547] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.
(APIServer pid=331487) [transformers] Qwen3VL video processing does not apply the per-frame pixel cap the reference implementation (qwen-vl-utils) applies, so some videos cost far more tokens than they would there. In v5.22 the capped behavior will become the default and `cap_pixels_per_frame` will be removed. Pass `cap_pixels_per_frame=True` to adopt the reference behavior now, or `False` to keep the current behavior and silence this warning.
(APIServer pid=331487) INFO 2026-09-22T04:13:04Z [base.py:244] Multi-modal warmup completed in 19.528s
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [base.py:244] Readonly multi-modal warmup completed in 1.277s
(APIServer pid=331487) WARNING 2026-09-22T04:13:05Z [model.py:1759] Default vLLM sampling parameters have been overridden by the model's `generation_config.json`: `{'temperature': 1.0, 'top_k': 20, 'top_p': 0.95}`. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [entry.py:139] Starting vLLM server on http://127.0.0.1:8000
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:61] Available routes are:
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /openapi.json, Methods: GET, HEAD
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /docs, Methods: GET, HEAD
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /docs/oauth2-redirect, Methods: GET, HEAD
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /redoc, Methods: GET, HEAD
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /load, Methods: GET
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /version, Methods: GET
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /health, Methods: GET
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /metrics, Methods: GET
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /tokenize, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /detokenize, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/models, Methods: GET
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /ping, Methods: GET
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /ping, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /invocations, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/chat/completions, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/chat/completions/batch, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/responses, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/completions, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/messages, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/messages/count_tokens, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /generative_scoring, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/chat/completions/render, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/messages/render, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/completions/render, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/chat/completions/derender, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /v1/completions/derender, Methods: POST
(APIServer pid=331487) INFO 2026-09-22T04:13:05Z [launcher.py:70] Route: /inference/v1/generate, Methods: POST
(APIServer pid=331487) INFO:     Started server process [331487]
(APIServer pid=331487) INFO:     Waiting for application startup.
(APIServer pid=331487) INFO:     Application startup complete.
(APIServer pid=331487) INFO:     127.0.0.1:58388 - "GET /v1/models HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58396 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58398 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58412 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58428 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58442 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58456 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58470 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58478 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58494 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58504 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58518 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58522 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58532 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58538 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58548 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58554 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58560 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58566 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58578 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58584 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58600 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58606 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58612 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58628 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58644 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58652 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58654 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58658 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58674 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58686 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58690 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58692 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58700 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58704 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58720 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58734 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58738 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58744 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58756 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58770 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58786 - "POST /tokenize HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:58790 - "POST /tokenize HTTP/1.1" 200 OK
(EngineCore pid=332027) WARNING 2026-09-22T04:13:19Z [jit_monitor.py:141] Triton kernel JIT compilation during inference: layer_norm_fwd_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(EngineCore pid=332027) WARNING 2026-09-22T04:13:19Z [jit_monitor.py:141] Triton kernel JIT compilation during inference: _topk_log_softmax_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(APIServer pid=331487) INFO:     127.0.0.1:40532 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41138 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41148 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41158 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41160 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41170 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41176 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41184 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41186 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41200 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41212 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41214 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41220 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41234 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41246 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41256 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41272 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41288 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41292 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41300 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41304 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41318 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41334 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41342 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41356 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41360 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41366 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41380 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41392 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41406 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41422 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41426 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41432 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41444 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41456 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41462 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41468 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41474 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41484 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41486 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41494 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41508 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41518 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41530 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41538 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41550 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41556 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41560 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41570 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41574 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41590 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41592 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41606 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41612 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41614 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41624 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41626 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41634 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41650 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41658 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41668 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41676 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41680 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41696 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41700 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41706 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41716 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41728 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41730 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41740 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41754 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41756 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41758 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41770 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41778 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41782 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:13:26Z [loggers.py:310] Engine 000: Avg prompt throughput: 2193.1 tokens/s, Avg generation throughput: 7.6 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=331487) INFO:     127.0.0.1:41794 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41810 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41820 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41834 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41840 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41848 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41850 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41852 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41862 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41870 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41874 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41882 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41892 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41908 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41916 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41918 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41930 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41940 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41944 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41950 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41964 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41980 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41992 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:41996 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42004 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42014 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42022 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42038 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42048 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42058 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42068 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:42084 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:13:36Z [loggers.py:310] Engine 000: Avg prompt throughput: 937.2 tokens/s, Avg generation throughput: 3.2 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=331487) INFO:     127.0.0.1:38036 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:38040 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:38042 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:38054 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:38064 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:13:46Z [loggers.py:310] Engine 000: Avg prompt throughput: 3983.4 tokens/s, Avg generation throughput: 0.5 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=331487) INFO:     127.0.0.1:38074 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:38078 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:38084 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:38092 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57330 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57334 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57344 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57360 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57376 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57392 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57402 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:13:56Z [loggers.py:310] Engine 000: Avg prompt throughput: 6782.3 tokens/s, Avg generation throughput: 1.1 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=331487) INFO:     127.0.0.1:57418 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57424 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57436 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:57444 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56300 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56316 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56322 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56336 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56352 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:14:06Z [loggers.py:310] Engine 000: Avg prompt throughput: 7002.8 tokens/s, Avg generation throughput: 0.9 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=331487) INFO:     127.0.0.1:56368 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56378 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56386 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:56400 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59378 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59394 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59400 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59404 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:14:16Z [loggers.py:310] Engine 000: Avg prompt throughput: 6128.0 tokens/s, Avg generation throughput: 0.8 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=331487) INFO:     127.0.0.1:59408 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59418 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59424 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59432 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59434 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59450 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59466 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59474 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59480 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59488 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59502 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59506 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59520 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59530 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:59534 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:45152 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:45156 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:45170 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:45184 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:45190 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:14:26Z [loggers.py:310] Engine 000: Avg prompt throughput: 2345.0 tokens/s, Avg generation throughput: 2.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 27.5%
(APIServer pid=331487) INFO:     127.0.0.1:39682 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39692 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39702 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39718 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39734 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39736 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39740 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39752 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39764 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39768 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39780 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39788 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39796 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39798 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39810 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39818 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39822 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39828 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39830 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39838 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39844 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39860 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39876 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:14:36Z [loggers.py:310] Engine 000: Avg prompt throughput: 2629.5 tokens/s, Avg generation throughput: 2.3 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 42.9%
(APIServer pid=331487) INFO:     127.0.0.1:39892 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39900 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39906 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39908 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39914 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39922 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39934 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39936 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39942 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39958 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39964 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO:     127.0.0.1:39974 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [launcher.py:124] [shutdown] API server: shutdown triggered
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [launcher.py:140] [shutdown] API server: stopping engine client mode=abort timeout=0s
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [core_client.py:689] [shutdown] MPClient: start timeout=0s
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [core_client.py:691] [shutdown] MPClient: stopping engine manager
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [utils.py:620] [shutdown] Process manager: send sigterm to process EngineCore
(APIServer pid=331487) WARNING 2026-09-22T04:14:38Z [utils.py:640] [shutdown] Process manager: force killing remaining processes count=1
(APIServer pid=331487) WARNING 2026-09-22T04:14:38Z [utils.py:645] [shutdown] Process manager: force killing remaining process EngineCore pid 332027
(EngineCore pid=332027) INFO 2026-09-22T04:14:38Z [core.py:1350] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=332027) INFO 2026-09-22T04:14:38Z [core.py:1501] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=332027) INFO 2026-09-22T04:14:38Z [core.py:1532] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=332027) INFO 2026-09-22T04:14:38Z [core.py:1363] [shutdown] EngineCore: exiting busy loop
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [core_client.py:693] [shutdown] MPClient: engine manager stopped
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [core_client.py:694] [shutdown] MPClient: cleaning up background resources
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [core_client.py:696] [shutdown] MPClient: complete
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [launcher.py:149] [shutdown] API server: engine client stopped
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [launcher.py:152] [shutdown] API server: signalling HTTP server shutdown
(APIServer pid=331487) INFO 2026-09-22T04:14:38Z [launcher.py:173] [shutdown] API server: shutting down FastAPI HTTP server
(APIServer pid=331487) INFO:     Shutting down
(APIServer pid=331487) INFO:     Waiting for application shutdown.
(APIServer pid=331487) INFO:     Application shutdown complete.
/workshop/.local/share/uv/python/cpython-3.12.14-linux-x86_64-gnu/lib/python3.12/multiprocessing/resource_tracker.py:279: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '