file-type

Java AtomicInteger实现多线程安全计数器

TXT文件

963B | 更新于2024-08-03 | 3 浏览量 | 0 下载量 举报 收藏
download 立即下载
在这个Java编程示例中,我们探讨了如何利用Java Memory Model(JMM,Java内存模型)来构建一个简单的线程安全计数器。标题"使用Java的Memory Model实现一个简单的计数器"着重于关键概念,即如何在并发编程中处理多线程环境下的数据同步问题。 Java的Memory Model规定了在多线程环境下,每个线程看到的共享变量的值是如何同步更新的。在多线程情况下,传统的同步机制如synchronized可能会引发竞态条件,导致数据不一致。而`AtomicInteger`类正是Java提供的一种特殊的类,其设计目标就是为了解决这个问题。 `AtomicInteger`类是`java.util.concurrent.atomic`包下的一个类,它实现了原子操作。原子操作是指一系列操作在单个操作单元中完成,不会被其他线程中断或观察到中间状态。它的核心原理是CAS(Compare and Swap)算法,这是一种低级别的同步机制,它尝试将某个值与目标变量进行比较,如果相等,则用新的值替换,否则不做任何操作。`incrementAndGet()`方法就是基于CAS实现的,它会原子性地将计数器加一,并返回新的值,确保在此过程中没有其他线程能读取到不完整的计数值。 在`Counter`类中,我们定义了一个私有静态成员变量`count`,类型为`AtomicInteger`,初始值设为0。`increment()`方法调用`incrementAndGet()`方法来增加计数器的值,而`getCount()`方法则通过`get()`方法获取并返回当前的计数器值。这两个方法由于`AtomicInteger`的原子性特性,确保了在多线程环境中,无论何时调用,都能得到正确的结果,且不会出现数据竞争。 总结来说,本代码展示了Java内存模型如何通过`AtomicInteger`类来确保在并发环境中的线程安全,使得多线程程序能够正确、高效地共享和更新数据。这对于理解和编写高并发场景下的代码至关重要,特别是在避免数据一致性问题时。同时,学习和掌握这些高级并发工具和技术,有助于提升程序的性能和可维护性。

相关推荐

filetype

“http_request_duration_highr_seconds_bucket{le="0.01"} : "4847.0" http_request_duration_highr_seconds_bucket{le="0.1"} : "5046.0" http_request_duration_highr_seconds_bucket{le="0.05"} : "4859.0" http_request_duration_highr_seconds_bucket{le="0.5"} : "5934.0" http_request_duration_highr_seconds_bucket{le="0.025"} : "4853.0" http_request_duration_highr_seconds_bucket{le="0.25"} : "5279.0" http_request_duration_highr_seconds_bucket{le="0.075"} : "4866.0" http_request_duration_highr_seconds_bucket{le="0.75"} : "6667.0" http_request_duration_highr_seconds_bucket{le="1.0"} : "7415.0" http_request_duration_highr_seconds_bucket{le="1.5"} : "8357.0" http_request_duration_highr_seconds_bucket{le="2.0"} : "9227.0" http_request_duration_highr_seconds_bucket{le="2.5"} : "10121.0" http_request_duration_highr_seconds_bucket{le="3.0"} : "10998.0" http_request_duration_highr_seconds_bucket{le="3.5"} : "11729.0" http_request_duration_highr_seconds_bucket{le="4.0"} : "12385.0" http_request_duration_highr_seconds_bucket{le="4.5"} : "12882.0" http_request_duration_highr_seconds_bucket{le="5.0"} : "13234.0" http_request_duration_highr_seconds_bucket{le="7.5"} : "14374.0" http_request_duration_highr_seconds_bucket{le="10.0"} : "15581.0" http_request_duration_highr_seconds_bucket{le="30.0"} : "25709.0" http_request_duration_highr_seconds_bucket{le="60.0"} : "26209.0" http_request_duration_highr_seconds_bucket{le="+Inf"} : "26448.0" http_request_duration_highr_seconds_count : "26448.0" http_request_duration_highr_seconds_created : "1.7512560890372858e+09" http_request_duration_highr_seconds_sum : "250360.7440119423" http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.1",method="POST"} : "227.0" http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="0.5",method="POST"} : "1115.0" http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="1.0",method="POST"} : "2596.0" http_request_duration_seconds_bucket{handler="/v1/chat/completions",le="+Inf",method="POST"} : "21629.0" http_request_duration_seconds_bucket{handler="/v1/models",le="0.1",method="GET"} : "1.0" http_request_duration_seconds_bucket{handler="/v1/models",le="0.5",method="GET"} : "1.0" http_request_duration_seconds_bucket{handler="/v1/models",le="1.0",method="GET"} : "1.0" http_request_duration_seconds_bucket{handler="/v1/models",le="+Inf",method="GET"} : "1.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="GET"} : "4693.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="HEAD"} : "6.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="OPTIONS"} : "12.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="POST"} : "95.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="PROPFIND"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="PUT"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="SEARCH"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="0.1",method="TRACE"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="GET"} : "4693.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="HEAD"} : "6.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="OPTIONS"} : "12.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="POST"} : "95.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="PROPFIND"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="PUT"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="SEARCH"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="0.5",method="TRACE"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="GET"} : "4693.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="HEAD"} : "6.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="OPTIONS"} : "12.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="POST"} : "95.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="PROPFIND"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="PUT"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="SEARCH"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="1.0",method="TRACE"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="GET"} : "4693.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="HEAD"} : "6.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="OPTIONS"} : "12.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="POST"} : "95.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="PROPFIND"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="PUT"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="SEARCH"} : "3.0" http_request_duration_seconds_bucket{handler="none",le="+Inf",method="TRACE"} : "3.0" http_request_duration_seconds_count{handler="/v1/chat/completions",method="POST"} : "21629.0" http_request_duration_seconds_count{handler="/v1/models",method="GET"} : "1.0" http_request_duration_seconds_count{handler="none",method="GET"} : "4693.0" http_request_duration_seconds_count{handler="none",method="HEAD"} : "6.0" http_request_duration_seconds_count{handler="none",method="OPTIONS"} : "12.0" http_request_duration_seconds_count{handler="none",method="POST"} : "95.0" http_request_duration_seconds_count{handler="none",method="PROPFIND"} : "3.0" http_request_duration_seconds_count{handler="none",method="PUT"} : "3.0" http_request_duration_seconds_count{handler="none",method="SEARCH"} : "3.0" http_request_duration_seconds_count{handler="none",method="TRACE"} : "3.0" http_request_duration_seconds_created{handler="/v1/chat/completions",method="POST"} : "1.7512560967123778e+09" http_request_duration_seconds_created{handler="/v1/models",method="GET"} : "1.7536925794242406e+09" http_request_duration_seconds_created{handler="none",method="GET"} : "1.7516341020108707e+09" http_request_duration_seconds_created{handler="none",method="HEAD"} : "1.751634176119915e+09" http_request_duration_seconds_created{handler="none",method="OPTIONS"} : "1.7516341579990425e+09" http_request_duration_seconds_created{handler="none",method="POST"} : "1.7516341771295128e+09" http_request_duration_seconds_created{handler="none",method="PROPFIND"} : "1.7516341696153226e+09" http_request_duration_seconds_created{handler="none",method="PUT"} : "1.7516349058165367e+09" http_request_duration_seconds_created{handler="none",method="SEARCH"} : "1.7516341693599503e+09" http_request_duration_seconds_created{handler="none",method="TRACE"} : "1.751634165566383e+09" http_request_duration_seconds_sum{handler="/v1/chat/completions",method="POST"} : "250359.90331811644" http_request_duration_seconds_sum{handler="/v1/models",method="GET"} : "0.0027880221605300903" http_request_duration_seconds_sum{handler="none",method="GET"} : "0.8171688430011272" http_request_duration_seconds_sum{handler="none",method="HEAD"} : "0.0009557865560054779" http_request_duration_seconds_sum{handler="none",method="OPTIONS"} : "0.0028338953852653503" http_request_duration_seconds_sum{handler="none",method="POST"} : "0.014691390097141266" http_request_duration_seconds_sum{handler="none",method="PROPFIND"} : "0.000380123034119606" http_request_duration_seconds_sum{handler="none",method="PUT"} : "0.00042458251118659973" http_request_duration_seconds_sum{handler="none",method="SEARCH"} : "0.0005713216960430145" http_request_duration_seconds_sum{handler="none",method="TRACE"} : "0.0008798614144325256" http_request_size_bytes_count{handler="/v1/chat/completions"} : "21629.0" http_request_size_bytes_count{handler="/v1/models"} : "1.0" http_request_size_bytes_count{handler="none"} : "4818.0" http_request_size_bytes_created{handler="/v1/chat/completions"} : "1.7512560967123284e+09" http_request_size_bytes_created{handler="/v1/models"} : "1.753692579424021e+09" http_request_size_bytes_created{handler="none"} : "1.7516341020104244e+09" http_request_size_bytes_sum{handler="/v1/chat/completions"} : "806031.0" http_request_size_bytes_sum{handler="/v1/models"} : "0.0" http_request_size_bytes_sum{handler="none"} : "32625.0" http_requests_created{handler="/v1/chat/completions",method="POST",status="2xx"} : "1.7512560967123055e+09" http_requests_created{handler="/v1/chat/completions",method="POST",status="4xx"} : "1.7514186825033803e+09" http_requests_created{handler="/v1/models",method="GET",status="2xx"} : "1.753692579423783e+09" http_requests_created{handler="none",method="GET",status="4xx"} : "1.7516341020101185e+09" http_requests_created{handler="none",method="HEAD",status="4xx"} : "1.7516341761198838e+09" http_requests_created{handler="none",method="OPTIONS",status="4xx"} : "1.7516341579990091e+09" http_requests_created{handler="none",method="POST",status="4xx"} : "1.7516341771294773e+09" http_requests_created{handler="none",method="PROPFIND",status="4xx"} : "1.7516341696152897e+09" http_requests_created{handler="none",method="PUT",status="4xx"} : "1.7516349058164842e+09" http_requests_created{handler="none",method="SEARCH",status="4xx"} : "1.7516341693599005e+09" http_requests_created{handler="none",method="TRACE",status="4xx"} : "1.7516341655663416e+09" http_requests_total{handler="/v1/chat/completions",method="POST",status="2xx"} : "21576.0" http_requests_total{handler="/v1/chat/completions",method="POST",status="4xx"} : "53.0" http_requests_total{handler="/v1/models",method="GET",status="2xx"} : "1.0" http_requests_total{handler="none",method="GET",status="4xx"} : "4693.0" http_requests_total{handler="none",method="HEAD",status="4xx"} : "6.0" http_requests_total{handler="none",method="OPTIONS",status="4xx"} : "12.0" http_requests_total{handler="none",method="POST",status="4xx"} : "95.0" http_requests_total{handler="none",method="PROPFIND",status="4xx"} : "3.0" http_requests_total{handler="none",method="PUT",status="4xx"} : "3.0" http_requests_total{handler="none",method="SEARCH",status="4xx"} : "3.0" http_requests_total{handler="none",method="TRACE",status="4xx"} : "3.0" http_response_size_bytes_count{handler="/v1/chat/completions"} : "21629.0" http_response_size_bytes_count{handler="/v1/models"} : "1.0" http_response_size_bytes_count{handler="none"} : "4818.0" http_response_size_bytes_created{handler="/v1/chat/completions"} : "1.7512560967123535e+09" http_response_size_bytes_created{handler="/v1/models"} : "1.7536925794240377e+09" http_response_size_bytes_created{handler="none"} : "1.751634102010456e+09" http_response_size_bytes_sum{handler="/v1/chat/completions"} : "3.541716e+06" http_response_size_bytes_sum{handler="/v1/models"} : "538.0" http_response_size_bytes_sum{handler="none"} : "105996.0" process_cpu_seconds_total : "2391.38" process_max_fds : "1.073741816e+09" process_open_fds : "48.0" process_resident_memory_bytes : "4.28490752e+08" process_start_time_seconds : "1.75125604907e+09" process_virtual_memory_bytes : "1.2146741248e+010" python_gc_collections_total{generation="0"} : "5127.0" python_gc_collections_total{generation="1"} : "465.0" python_gc_collections_total{generation="2"} : "29.0" python_gc_objects_collected_total{generation="0"} : "8032.0" python_gc_objects_collected_total{generation="1"} : "1350.0" python_gc_objects_collected_total{generation="2"} : "994.0" python_gc_objects_uncollectable_total{generation="0"} : "0.0" python_gc_objects_uncollectable_total{generation="1"} : "0.0" python_gc_objects_uncollectable_total{generation="2"} : "0.0" python_info{implementation="CPython",major="3",minor="12",patchlevel="10",version="3.12.10"} : "1.0" vllm:cache_config_info{block_size="16",cache_dtype="auto",calculate_kv_scales="False",cpu_offload_gb="0",enable_prefix_caching="True",gpu_memory_utilization="0.95",is_attention_free="False",num_gpu_blocks_override="None",prefix_caching_hash_algo="builtin",sliding_window="None",swap_space="4",swap_space_bytes="4294967296"} : "1.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.3",model_name="qwen2.5-72b-instruct-gptq-int4"} : "491.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1088.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="0.8",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1947.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2563.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="1.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "3503.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "4373.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="2.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5264.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "8367.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "10710.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="15.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "15758.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "19765.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="30.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20837.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="40.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21082.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21252.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="60.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21336.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="120.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21514.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="240.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21560.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="480.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21563.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="960.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21565.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="1920.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="7680.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:e2e_request_latency_seconds_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:e2e_request_latency_seconds_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:e2e_request_latency_seconds_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660305371e+09" vllm:e2e_request_latency_seconds_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "249588.05839586258" vllm:generation_tokens_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660302482e+09" vllm:generation_tokens_total{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.146206e+06" vllm:gpu_cache_usage_perc{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.008373786407766981" vllm:gpu_prefix_cache_hits_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.751256066030216e+09" vllm:gpu_prefix_cache_hits_total{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "616813.0" vllm:gpu_prefix_cache_queries_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660302012e+09" vllm:gpu_prefix_cache_queries_total{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.150983e+06" vllm:iteration_tokens_total_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.262281e+06" vllm:iteration_tokens_total_bucket{engine="0",le="8.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.111395e+06" vllm:iteration_tokens_total_bucket{engine="0",le="16.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.166718e+06" vllm:iteration_tokens_total_bucket{engine="0",le="32.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.167375e+06" vllm:iteration_tokens_total_bucket{engine="0",le="64.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.167422e+06" vllm:iteration_tokens_total_bucket{engine="0",le="128.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.167871e+06" vllm:iteration_tokens_total_bucket{engine="0",le="256.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.169028e+06" vllm:iteration_tokens_total_bucket{engine="0",le="512.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.182286e+06" vllm:iteration_tokens_total_bucket{engine="0",le="1024.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.185063e+06" vllm:iteration_tokens_total_bucket{engine="0",le="2048.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.186771e+06" vllm:iteration_tokens_total_bucket{engine="0",le="4096.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.187694e+06" vllm:iteration_tokens_total_bucket{engine="0",le="8192.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.188073e+06" vllm:iteration_tokens_total_bucket{engine="0",le="16384.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.188296e+06" vllm:iteration_tokens_total_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.188353e+06" vllm:iteration_tokens_total_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.188353e+06" vllm:iteration_tokens_total_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660303833e+09" vllm:iteration_tokens_total_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.3710533e+07" vllm:num_preemptions_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.751256066030228e+09" vllm:num_preemptions_total{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:num_requests_running{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.0" vllm:num_requests_waiting{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:prompt_tokens_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660302384e+09" vllm:prompt_tokens_total{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.8564327e+07" vllm:request_decode_time_seconds_bucket{engine="0",le="0.3",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1036.0" vllm:request_decode_time_seconds_bucket{engine="0",le="0.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1763.0" vllm:request_decode_time_seconds_bucket{engine="0",le="0.8",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2659.0" vllm:request_decode_time_seconds_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "3215.0" vllm:request_decode_time_seconds_bucket{engine="0",le="1.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "3962.0" vllm:request_decode_time_seconds_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "4894.0" vllm:request_decode_time_seconds_bucket{engine="0",le="2.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5906.0" vllm:request_decode_time_seconds_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "8900.0" vllm:request_decode_time_seconds_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "11080.0" vllm:request_decode_time_seconds_bucket{engine="0",le="15.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "16114.0" vllm:request_decode_time_seconds_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20016.0" vllm:request_decode_time_seconds_bucket{engine="0",le="30.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20972.0" vllm:request_decode_time_seconds_bucket{engine="0",le="40.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21180.0" vllm:request_decode_time_seconds_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21317.0" vllm:request_decode_time_seconds_bucket{engine="0",le="60.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21386.0" vllm:request_decode_time_seconds_bucket{engine="0",le="120.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21521.0" vllm:request_decode_time_seconds_bucket{engine="0",le="240.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21560.0" vllm:request_decode_time_seconds_bucket{engine="0",le="480.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21563.0" vllm:request_decode_time_seconds_bucket{engine="0",le="960.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21565.0" vllm:request_decode_time_seconds_bucket{engine="0",le="1920.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_decode_time_seconds_bucket{engine="0",le="7680.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_decode_time_seconds_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_decode_time_seconds_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_decode_time_seconds_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660307164e+09" vllm:request_decode_time_seconds_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "235893.4257239038" vllm:request_generation_tokens_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_generation_tokens_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "375.0" vllm:request_generation_tokens_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "680.0" vllm:request_generation_tokens_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1406.0" vllm:request_generation_tokens_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2613.0" vllm:request_generation_tokens_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "4881.0" vllm:request_generation_tokens_bucket{engine="0",le="100.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "8468.0" vllm:request_generation_tokens_bucket{engine="0",le="200.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "10587.0" vllm:request_generation_tokens_bucket{engine="0",le="500.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20580.0" vllm:request_generation_tokens_bucket{engine="0",le="1000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21278.0" vllm:request_generation_tokens_bucket{engine="0",le="2000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21486.0" vllm:request_generation_tokens_bucket{engine="0",le="5000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21561.0" vllm:request_generation_tokens_bucket{engine="0",le="10000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21563.0" vllm:request_generation_tokens_bucket{engine="0",le="20000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21565.0" vllm:request_generation_tokens_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_generation_tokens_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_generation_tokens_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660303566e+09" vllm:request_generation_tokens_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.131528e+06" vllm:request_inference_time_seconds_bucket{engine="0",le="0.3",model_name="qwen2.5-72b-instruct-gptq-int4"} : "497.0" vllm:request_inference_time_seconds_bucket{engine="0",le="0.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1113.0" vllm:request_inference_time_seconds_bucket{engine="0",le="0.8",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1963.0" vllm:request_inference_time_seconds_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2577.0" vllm:request_inference_time_seconds_bucket{engine="0",le="1.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "3516.0" vllm:request_inference_time_seconds_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "4394.0" vllm:request_inference_time_seconds_bucket{engine="0",le="2.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5300.0" vllm:request_inference_time_seconds_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "8410.0" vllm:request_inference_time_seconds_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "10739.0" vllm:request_inference_time_seconds_bucket{engine="0",le="15.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "15812.0" vllm:request_inference_time_seconds_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "19801.0" vllm:request_inference_time_seconds_bucket{engine="0",le="30.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20861.0" vllm:request_inference_time_seconds_bucket{engine="0",le="40.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21104.0" vllm:request_inference_time_seconds_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21270.0" vllm:request_inference_time_seconds_bucket{engine="0",le="60.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21345.0" vllm:request_inference_time_seconds_bucket{engine="0",le="120.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21515.0" vllm:request_inference_time_seconds_bucket{engine="0",le="240.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21560.0" vllm:request_inference_time_seconds_bucket{engine="0",le="480.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21563.0" vllm:request_inference_time_seconds_bucket{engine="0",le="960.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21565.0" vllm:request_inference_time_seconds_bucket{engine="0",le="1920.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_inference_time_seconds_bucket{engine="0",le="7680.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_inference_time_seconds_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_inference_time_seconds_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_inference_time_seconds_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660305977e+09" vllm:request_inference_time_seconds_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "247843.74306567665" vllm:request_max_num_generation_tokens_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "375.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "680.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1406.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2613.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "4881.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="100.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "8468.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="200.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "10587.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="500.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20580.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="1000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21278.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="2000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21486.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="5000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21561.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="10000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21563.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="20000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21565.0" vllm:request_max_num_generation_tokens_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_max_num_generation_tokens_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_max_num_generation_tokens_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.751256066030409e+09" vllm:request_max_num_generation_tokens_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.131528e+06" vllm:request_params_max_tokens_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_params_max_tokens_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_params_max_tokens_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_params_max_tokens_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_params_max_tokens_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_params_max_tokens_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_params_max_tokens_bucket{engine="0",le="100.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.0" vllm:request_params_max_tokens_bucket{engine="0",le="200.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2.0" vllm:request_params_max_tokens_bucket{engine="0",le="500.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "4.0" vllm:request_params_max_tokens_bucket{engine="0",le="1000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "7.0" vllm:request_params_max_tokens_bucket{engine="0",le="2000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "136.0" vllm:request_params_max_tokens_bucket{engine="0",le="5000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "422.0" vllm:request_params_max_tokens_bucket{engine="0",le="10000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "439.0" vllm:request_params_max_tokens_bucket{engine="0",le="20000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "924.0" vllm:request_params_max_tokens_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_max_tokens_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_max_tokens_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660304518e+09" vllm:request_params_max_tokens_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "6.26237766e+08" vllm:request_params_n_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_n_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_n_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_n_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_n_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_n_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_n_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_params_n_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660304315e+09" vllm:request_params_n_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="0.3",model_name="qwen2.5-72b-instruct-gptq-int4"} : "17877.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="0.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "18517.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="0.8",model_name="qwen2.5-72b-instruct-gptq-int4"} : "19080.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "19493.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="1.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20099.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20423.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="2.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20583.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21096.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21308.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="15.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21468.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21525.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="30.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21539.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="40.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21565.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="60.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="120.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="240.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="480.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="960.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="1920.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="7680.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prefill_time_seconds_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.751256066030625e+09" vllm:request_prefill_time_seconds_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "11950.31734177284" vllm:request_prompt_tokens_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_prompt_tokens_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_prompt_tokens_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_prompt_tokens_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_prompt_tokens_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_prompt_tokens_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "52.0" vllm:request_prompt_tokens_bucket{engine="0",le="100.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "110.0" vllm:request_prompt_tokens_bucket{engine="0",le="200.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1287.0" vllm:request_prompt_tokens_bucket{engine="0",le="500.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "15764.0" vllm:request_prompt_tokens_bucket{engine="0",le="1000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "18257.0" vllm:request_prompt_tokens_bucket{engine="0",le="2000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "19958.0" vllm:request_prompt_tokens_bucket{engine="0",le="5000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21066.0" vllm:request_prompt_tokens_bucket{engine="0",le="10000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21416.0" vllm:request_prompt_tokens_bucket{engine="0",le="20000.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21522.0" vllm:request_prompt_tokens_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prompt_tokens_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_prompt_tokens_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660303123e+09" vllm:request_prompt_tokens_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.8563094e+07" vllm:request_queue_time_seconds_bucket{engine="0",le="0.3",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21489.0" vllm:request_queue_time_seconds_bucket{engine="0",le="0.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21489.0" vllm:request_queue_time_seconds_bucket{engine="0",le="0.8",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21489.0" vllm:request_queue_time_seconds_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21489.0" vllm:request_queue_time_seconds_bucket{engine="0",le="1.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21489.0" vllm:request_queue_time_seconds_bucket{engine="0",le="2.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21489.0" vllm:request_queue_time_seconds_bucket{engine="0",le="2.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21497.0" vllm:request_queue_time_seconds_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21519.0" vllm:request_queue_time_seconds_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21531.0" vllm:request_queue_time_seconds_bucket{engine="0",le="15.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21541.0" vllm:request_queue_time_seconds_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21550.0" vllm:request_queue_time_seconds_bucket{engine="0",le="30.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21568.0" vllm:request_queue_time_seconds_bucket{engine="0",le="40.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="50.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="60.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="120.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="240.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="480.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="960.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="1920.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="7680.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21574.0" vllm:request_queue_time_seconds_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660305698e+09" vllm:request_queue_time_seconds_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1095.8753950130194" vllm:request_success_created{engine="0",finished_reason="abort",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660302787e+09" vllm:request_success_created{engine="0",finished_reason="length",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.751256066030273e+09" vllm:request_success_created{engine="0",finished_reason="stop",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660302663e+09" vllm:request_success_total{engine="0",finished_reason="abort",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:request_success_total{engine="0",finished_reason="length",model_name="qwen2.5-72b-instruct-gptq-int4"} : "16.0" vllm:request_success_total{engine="0",finished_reason="stop",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21558.0" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.01",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.1",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.115606e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.2",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.120195e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.3",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.120706e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.4",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12097e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.05",model_name="qwen2.5-72b-instruct-gptq-int4"} : "4.473164e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.121215e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.15",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.119781e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.025",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.075",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.096754e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="0.75",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.121683e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.122067e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="2.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.123599e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="7.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="40.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="80.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5.12463e+06" vllm:time_per_output_token_seconds_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660305083e+09" vllm:time_per_output_token_seconds_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "236493.54754120763" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.001",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.01",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.1",model_name="qwen2.5-72b-instruct-gptq-int4"} : "10761.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.02",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.04",model_name="qwen2.5-72b-instruct-gptq-int4"} : "22.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.005",model_name="qwen2.5-72b-instruct-gptq-int4"} : "0.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "18421.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.06",model_name="qwen2.5-72b-instruct-gptq-int4"} : "2307.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.08",model_name="qwen2.5-72b-instruct-gptq-int4"} : "5692.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.25",model_name="qwen2.5-72b-instruct-gptq-int4"} : "17492.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="0.75",model_name="qwen2.5-72b-instruct-gptq-int4"} : "18898.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="1.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "19414.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="2.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "20526.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="5.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21039.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="7.5",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21181.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="10.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21258.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="20.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21496.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="40.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21561.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="80.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21576.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="160.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21576.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="640.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21576.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="2560.0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21576.0" vllm:time_to_first_token_seconds_bucket{engine="0",le="+Inf",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21576.0" vllm:time_to_first_token_seconds_count{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "21576.0" vllm:time_to_first_token_seconds_created{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "1.7512560660304754e+09" vllm:time_to_first_token_seconds_sum{engine="0",model_name="qwen2.5-72b-instruct-gptq-int4"} : "13699.205335855484" ”可以根据这些数据推测出或者计算出“QPS”“最大运行数”"最大等待数""失败率""成功率""平均耗时(ms)"吗

小兔子平安
  • 粉丝: 303
上传资源 快速赚钱