All Articles

Vegeta Notes

1. 工具选择

压力测试也是后端工作中比较多见的需求。一般来说，针对简单的HTTP请求，用得比较多的是：ab - Apache HTTP server benchmarking tool & wg/wrk。这两款都是使用C语言进行编写的工具，性能有十分保障。

但日常工作中，我们的压测还有一些灵活的需求：

  • 压测的目标不一定是单个的API
  • 压测的时长和速率需要能调节
  • 压测的结果要能快速展示，包括文字和graph两种形式
  • 压测的结果要能进行存储，以使用其他软件进行拓展研究
  • 压测要能以client端分布式的方式进行，以保证在制造巨量的请求时，可以使用多台client进行联合测试，且测试结果可以融合归一进行分析

以上的几点需求，在ab和wrk中都不能完全满足，特别是分布式测试这点，基本上没见过市面上有哪个软件能够很好做到。

这里就要介绍下今天的主角了：tsenart/vegeta。这款工具是使用golang编写的压测工具，由于golang非常优秀的goroutine，这款工具的表现几乎不逊色于c语言编写的几位前辈。

安装：

$ brew update && brew install vegeta

2. 后端示例

本文会进行一些范例演示，所有的vegeta命令所针对的后端服务器都使用如下代码：

import * as express from 'express';

const app = express();
const port = 5000;

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

app.get('/get', async function (req, res) {
  const data = {
    method: 'GET',
    path: '/get',
    query: req.query,
    headers: req.headers
  };
  console.log(JSON.stringify(data));
  res.json(data);
});

app.post('/post', async function (req, res) {
  const data = {
    method: 'POST',
    path: '/post',
    data: req.body,
    headers: req.headers
  };
  console.log(JSON.stringify(data));
  res.json(data);
});

app.listen(port, () => {
  console.log(`Server listening at http://localhost:${port}`);
});

该服务器仅只有两个API：

  • http://localhost:5000/get
  • http://localhost:5000/post

这两个API会在收到请求后，在console中打印出该请求的所有信息。

3. vegeta command

vegeta命令包含多个功能，在运行的时候，需要额外提供一个子命令，来明确目前运行的时候具体要做什么：

  • attack：压测命令
  • encode：转码命令，将压测输出的结果集从默认格式转成其他的格式（json等）
  • plot：graph命令，将压测输出的结果生成graph html，以供查看
  • report：报告命令，将压测输出的结果生成文字形式的report

vegeta使用*nix pipeline非常重，很多参数的输入和结果的输出都默认使用stdin和stdout，所以非常适合多个命令进行前后管道顺序执行。

另外需要提一点，vegeta命令中提供的所有文件的路径，都必须以当前的工作路径（cwd）为基准。

e.g

$ echo 'GET ...' | vegeta attack ... | vegeta report

4. vegeta attack

该命令用以发起压测，官方文档：link。有几个option需要简单说明下：

  • -targets：该option指定attack的目标是什么，默认读取stdin的输入，也可以提供一个文件路径，让vegeta读取其文本内容来获取目标
  • -format：该option一般使用http，表示target给的内容必须按RFC 2616的格式提供：'GET http://localhost:5000/...'
  • -header：该option可以提供多个，每个option表示额外提供一个header
  • -body：该option用来指定post body的数据文件，注意，如果在targets里也提供了@/body/file/path则以target里的为准，当前option提供的body会被忽略
  • -timeout：该option表示每个http请求的超时时长，默认为0表示无超时
  • -workers：该option用来指定vegeta刚启动时需要启动的workers数量，后续实际workers数值会根据需求自行上升以满足rate的要求
  • -max-workers：该option用来限制attack时的最高并发数量
  • -duration：该option用来控制测试时长，使用字符串形式来提供数值：5s
  • -rate：该option用来控制请求并发速率
    • 5/s表示每秒vegeta会一共发出5个请求（vegeta会启动补充worker，如果速度设置很高的话）
    • 设置为0表示无限制，vegeta会尽可能快地发出请求
    • 设置为0的时候需要和max-workers配合使用，来行成一个固定的并发速率，否则可能会快速耗尽系统资源

4.1 vegeta attack: target from stdin

如下命令启动5秒的测试，并每秒发送5个请求，最后以文字形式输出报告。

  • vegeta attack从自己的stdin读取echo的字符串作为target进行压测
  • vegeta report从自己的stdin读取上一个命令vegeta attack的stdout，来获取数据进行处理
$ echo 'GET http://localhost:5000/get?get_key=get_val' | \
    vegeta attack \
        -duration=5s \
        -rate=5/s \
        -format=http \
        -header 'Cache-Control: no-cache' | \
            vegeta report -type=text

服务端输出：

Server listening at http://localhost:5000
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"0","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"1","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"2","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"3","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"4","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"5","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"6","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"7","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"8","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"9","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"10","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"11","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"12","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"13","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"14","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"15","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"16","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"17","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"18","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"19","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"20","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"21","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"22","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"23","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-vegeta-seq":"24","accept-encoding":"gzip"}}

report：

Requests      [total, rate, throughput]         25, 5.21, 5.21
Duration      [total, attack, wait]             4.799s, 4.798s, 1.073ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.023ms, 1.57ms, 1.386ms, 1.97ms, 2.819ms, 4.442ms, 4.442ms
Bytes In      [total, mean]                     6762, 270.48
Bytes Out     [total, mean]                     736, 29.44
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:25  
Error Set:

4.2 vegeta attack: target from file

如下命令会启动持续5秒的测试，并每秒发送5个请求，请求内容来自targets.txt文件的内容，最后以文字形式输出报：

vegeta attack \
    -targets=./vegeta/targets.txt \
    -duration=5s \
    -rate=5/s \
    -format=http \
    -header 'Cache-Control: no-cache' | \
        vegeta report -type=text

targets.txt

GET http://localhost:5000/get?get_key=get_val
X-Add-Get-ID1: 78
X-Add-Get-ID2: 88

POST http://localhost:5000/post?post_key1=post_val
X-Add-Post-ID: 199
Content-Type: application/json
@./vegeta/postdata.json

POST http://localhost:5000/post?post_key2=post_val
X-Add-Post-ID: 299
Content-Type: application/json
@./vegeta/postdata.json

postdata.json

{
  "postdata": {
    "a": 1,
    "b": 2
  }
}
  • vegeta发送的请求会以targets.txt内列的target一个个请求发送下去
  • 每个target下面可以附带额外的header，不限数量，每行一个header
  • POST的target可以在下面带上一行@/body/file/path格式的文本，表示读取该位置的文件，作为post的body使用

服务端输出：

Server listening at http://localhost:5000
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"0","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"1","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"2","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"3","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"4","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"5","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"6","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"7","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"8","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"9","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"10","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"11","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"12","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"13","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"14","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"15","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"16","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"17","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"18","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"19","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"20","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"21","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"199","x-vegeta-seq":"22","accept-encoding":"gzip"}}
{"method":"POST","path":"/post","data":{"postdata":{"a":1,"b":2}},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","content-length":"46","cache-control":"no-cache","content-type":"application/json","x-add-post-id":"299","x-vegeta-seq":"23","accept-encoding":"gzip"}}
{"method":"GET","path":"/get","query":{"get_key":"get_val"},"headers":{"host":"localhost:5000","user-agent":"Go-http-client/1.1","cache-control":"no-cache","x-add-get-id1":"78","x-add-get-id2":"88","x-vegeta-seq":"24","accept-encoding":"gzip"}}

可以看到，根据我们的option设置，一共是25个请求5s * 5/s。然后请求和targets.txt里指定的一样，每次都是按1个GET2个POST这样的顺序发送。3*8=24之后，正好还有一个待发的，因此最后就是一个GET。

report：

Requests      [total, rate, throughput]         25, 5.21, 5.21
Duration      [total, attack, wait]             4.802s, 4.801s, 1.344ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.093ms, 1.687ms, 1.379ms, 1.769ms, 3.979ms, 5.077ms, 5.077ms
Bytes In      [total, mean]                     6762, 270.48
Bytes Out     [total, mean]                     736, 29.44
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:25  
Error Set:

5. vegeta plot

该命令用来将测试结果输出成graph的html，官方文档：link。option基本不需要调整。这里需要注意，输出的graph只含latency，不含其它信息。

e.g

$ echo "GET http://:80" | vegeta attack -name=50qps -rate=50 -duration=5s > results.50qps.bin
$ cat results.50qps.bin | vegeta plot > plot.50qps.html
$ echo "GET http://:80" | vegeta attack -name=100qps -rate=100 -duration=5s > results.100qps.bin
$ vegeta plot results.50qps.bin results.100qps.bin > plot.html

6. vegeta report

该明星用来将测试结果转换为报告，官方文档：link。

options：

  • --type：报告类型：text | json | hist[buckets] | hdrplot
  • --every：报告间隔，设置该值会在测试中也渐次输出report，e.g --every=1s

e.g

$ echo "GET http://:80" | vegeta attack -rate=10/s > results.gob
$ echo "GET http://:80" | vegeta attack -rate=100/s | vegeta encode > results.json
$ vegeta report results.*

7. vegeta encode

该命令用来将测试结果转换成其他格式，默认输出自vegeta attack的格式是gob，官方文档：link。

可以转换为：

  • gob
  • json
  • csv

使用option--to来进行指定。

e.g

$ echo "GET http://:80" | vegeta attack -rate=1/s > results.gob
$ cat results.gob | vegeta encode | jq -c 'del(.body)' | vegeta encode -to gob

8. Others

8.1 动态targets

根据官方instruction Usage: Generated targets，可以动态生成压测的target。另外也可以通过vegeta attack -targets=...的方式提供测试目标文件，然后在该文件中列出期望的用户行为来进行拟真测试。

8.2 分布式压测

这也是在开篇的时候提到的比较重要的一点需求：Usage: Distributed attacks。可以按该官方指引在多台client上进行分布式压测，vegeta有一点很好的是可以组合各个client生成出来的gob文件，来生成一份最终的结果report。

8.3 压测实时分析

Usage: Real-time Analysis，可以查看压测实时的运行状态。

EOF

Published 2021/9/11

Some tech & personal blog posts