All Articles

Vegeta Notes

1. 工具选择

压力测试也是后端工作中比较多见的需求。一般来说,针对简单的HTTP请求,用得比较多的是:ab - Apache HTTP server benchmarking tool & wg/wrk。这两款都是使用C语言进行编写的工具,性能有十分保障。

但日常工作中,我们的压测还有一些灵活的需求:

  • 压测的目标不一定是单个的API
  • 压测的时长和速率需要能调节
  • 压测的结果要能快速展示,包括文字和graph两种形式
  • 压测的结果要能进行存储,以使用其他软件进行拓展研究
  • 压测要能以client端分布式的方式进行,以保证在制造巨量的请求时,可以使用多台client进行联合测试,且测试结果可以融合归一进行分析

以上的几点需求,在ab和wrk中都不能完全满足,特别是分布式测试这点,基本上没见过市面上有哪个软件能够很好做到。

这里就要介绍下今天的主角了:tsenart/vegeta。这款工具是使用golang编写的压测工具,由于golang非常优秀的goroutine,这款工具的表现几乎不逊色于c语言编写的几位前辈。

安装:

$ brew update && brew install vegeta

2. 后端示例

本文会进行一些范例演示,所有的vegeta命令所针对的后端服务器都使用如下代码:

import * as express from 'express';

const app = express();
const port = 5000;

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

app.get('/get', async function (req, res) {
  const data = {
    method: 'GET',
    path: '/get',
    query: req.query,
    headers: req.headers
  };
  console.log(JSON.stringify(data));
  res.json(data);
});

app.post('/post', async function (req, res) {
  const data = {
    method: 'POST',
    path: '/post',
    data: req.body,
    headers: req.headers
  };
  console.log(JSON.stringify(data));
  res.json(data);
});

app.listen(port, () => {
  console.log(`Server listening at http://localhost:${port}`);
});

该服务器仅只有两个API:

  • http://localhost:5000/get
  • http://localhost:5000/post

这两个API会在收到请求后,在console中打印出该请求的所有信息。

3. vegeta command

vegeta命令包含多个功能,在运行的时候,需要额外提供一个子命令,来明确目前运行的时候具体要做什么:

  • attack:压测命令
  • encode:转码命令,将压测输出的结果集从默认格式转成其他的格式(json等)
  • plot:graph命令,将压测输出的结果生成graph html,以供查看
  • report:报告命令,将压测输出的结果生成文字形式的report

vegeta使用*nix pipeline非常重,很多参数的输入和结果的输出都默认使用stdinstdout,所以非常适合多个命令进行前后管道顺序执行。

另外需要提一点,vegeta命令中提供的所有文件的路径,都必须以当前的工作路径(cwd)为基准。

e.g

$ echo 'GET ...' | vegeta attack ... | vegeta report

4. vegeta attack

该命令用以发起压测,官方文档:link。有几个option需要简单说明下:

  • -targets:该option指定attack的目标是什么,默认读取stdin的输入,也可以提供一个文件路径,让vegeta读取其文本内容来获取目标
  • -format:该option一般使用http,表示target给的内容必须按RFC 2616的格式提供:'GET http://localhost:5000/...'
  • -header:该option可以提供多个,每个option表示额外提供一个header
  • -body:该option用来指定post body的数据文件,注意,如果在targets里也提供了@/body/file/path则以target里的为准,当前option提供的body会被忽略
  • -timeout:该option表示每个http请求的超时时长,默认为0表示无超时
  • -workers:该option用来指定vegeta刚启动时需要启动的workers数量,后续实际workers数值会根据需求自行上升以满足rate的要求
  • -max-workers:该option用来限制attack时的最高并发数量
  • -duration:该option用来控制测试时长,使用字符串形式来提供数值:5s
  • -rate:该option用来控制请求并发速率
    • 5/s表示每秒vegeta会一共发出5个请求(vegeta会启动补充worker,如果速度设置很高的话)
    • 设置为0表示无限制,vegeta会尽可能快地发出请求
    • 设置为0的时候需要和max-workers配合使用,来行成一个固定的并发速率,否则可能会快速耗尽系统资源

4.1 vegeta attack: target from stdin

如下命令启动5秒的测试,并每秒发送5个请求,最后以文字形式输出报告。

  • vegeta attack从自己的stdin读取echo的字符串作为target进行压测
  • vegeta report从自己的stdin读取上一个命令vegeta attack的stdout,来获取数据进行处理
$ echo 'GET http://localhost:5000/get?get_key=get_val' | \
    vegeta attack \
        -duration=5s \
        -rate=5/s \
        -format=http \
        -header 'Cache-Control: no-cache' | \
            vegeta report -type=text

服务端输出:

Server listening at http://localhost:5000
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"0","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"1","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"2","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"3","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"4","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"5","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"6","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"7","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"8","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"9","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"10","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"11","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"12","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"13","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"14","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"15","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"16","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"17","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"18","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"19","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"20","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"21","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"22","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"23","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"24","accept-encoding":"gzip"}}

report:

Requests      [total, rate, throughput]         25, 5.21, 5.21
Duration      [total, attack, wait]             4.799s, 4.798s, 1.073ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.023ms, 1.57ms, 1.386ms, 1.97ms, 2.819ms, 4.442ms, 4.442ms
Bytes In      [total, mean]                     6762, 270.48
Bytes Out     [total, mean]                     736, 29.44
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:25  
Error Set:

4.2 vegeta attack: target from file

如下命令会启动持续5秒的测试,并每秒发送5个请求,请求内容来自targets.txt文件的内容,最后以文字形式输出报:

vegeta attack \
    -targets=./vegeta/targets.txt \
    -duration=5s \
    -rate=5/s \
    -format=http \
    -header 'Cache-Control: no-cache' | \
        vegeta report -type=text

targets.txt

GET http://localhost:5000/get?get_key=get_val
X-Add-Get-ID1: 78
X-Add-Get-ID2: 88

POST http://localhost:5000/post?post_key1=post_val
X-Add-Post-ID: 199
Content-Type: application/json
@./vegeta/postdata.json

POST http://localhost:5000/post?post_key2=post_val
X-Add-Post-ID: 299
Content-Type: application/json
@./vegeta/postdata.json

postdata.json

{
  "postdata": {
    "a": 1,
    "b": 2
  }
}
  • vegeta发送的请求会以targets.txt内列的target一个个请求发送下去
  • 每个target下面可以附带额外的header,不限数量,每行一个header
  • POST的target可以在下面带上一行@/body/file/path格式的文本,表示读取该位置的文件,作为post的body使用

服务端输出:

Server listening at http://localhost:5000
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"0","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"1","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"2","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"3","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"4","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"5","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"6","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"7","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"8","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"9","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"10","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"11","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"12","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"13","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"14","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"15","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"16","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"17","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"18","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"19","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"20","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"21","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"22","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"23","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"24","accept-encoding":"gzip"}}

可以看到,根据我们的option设置,一共是25个请求5s * 5/s。然后请求和targets.txt里指定的一样,每次都是按1个GET2个POST这样的顺序发送。3*8=24之后,正好还有一个待发的,因此最后就是一个GET。

report:

Requests      [total, rate, throughput]         25, 5.21, 5.21
Duration      [total, attack, wait]             4.802s, 4.801s, 1.344ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.093ms, 1.687ms, 1.379ms, 1.769ms, 3.979ms, 5.077ms, 5.077ms
Bytes In      [total, mean]                     6762, 270.48
Bytes Out     [total, mean]                     736, 29.44
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:25  
Error Set:

5. vegeta plot

该命令用来将测试结果输出成graph的html,官方文档:link。option基本不需要调整。这里需要注意,输出的graph只含latency,不含其它信息。

e.g

$ echo "GET http://:80" | vegeta attack -name=50qps -rate=50 -duration=5s > results.50qps.bin
$ cat results.50qps.bin | vegeta plot > plot.50qps.html
$ echo "GET http://:80" | vegeta attack -name=100qps -rate=100 -duration=5s > results.100qps.bin
$ vegeta plot results.50qps.bin results.100qps.bin > plot.html

6. vegeta report

该明星用来将测试结果转换为报告,官方文档:link

options:

  • --type:报告类型:text | json | hist[buckets] | hdrplot
  • --every:报告间隔,设置该值会在测试中也渐次输出report,e.g --every=1s

e.g

$ echo "GET http://:80" | vegeta attack -rate=10/s > results.gob
$ echo "GET http://:80" | vegeta attack -rate=100/s | vegeta encode > results.json
$ vegeta report results.*

7. vegeta encode

该命令用来将测试结果转换成其他格式,默认输出自vegeta attack的格式是gob,官方文档:link

可以转换为:

  • gob
  • json
  • csv

使用option--to来进行指定。

e.g

$ echo "GET http://:80" | vegeta attack -rate=1/s > results.gob
$ cat results.gob | vegeta encode | jq -c 'del(.body)' | vegeta encode -to gob

8. Others

8.1 动态targets

根据官方instruction Usage: Generated targets,可以动态生成压测的target。另外也可以通过vegeta attack -targets=...的方式提供测试目标文件,然后在该文件中列出期望的用户行为来进行拟真测试。

8.2 分布式压测

这也是在开篇的时候提到的比较重要的一点需求:Usage: Distributed attacks。可以按该官方指引在多台client上进行分布式压测,vegeta有一点很好的是可以组合各个client生成出来的gob文件,来生成一份最终的结果report。

8.3 压测实时分析

Usage: Real-time Analysis,可以查看压测实时的运行状态。

EOF

Published 2021/9/11

Some tech & personal blog posts