X-Git-Url: https://git.m6w6.name/?a=blobdiff_plain;f=docs%2Fsource%2Fbin%2Fmemaslap.rst;h=6ac3e21a91038e1f661f45cb0df011eb27eed6ce;hb=HEAD;hp=eae05d813da08f0384d52b599b02f4eb887ac3ec;hpb=49aa94b8c4df6c3fd9d5c9688651c6f928f9aad1;p=awesomized%2Flibmemcached diff --git a/docs/source/bin/memaslap.rst b/docs/source/bin/memaslap.rst index eae05d81..6ac3e21a 100644 --- a/docs/source/bin/memaslap.rst +++ b/docs/source/bin/memaslap.rst @@ -2,24 +2,21 @@ memaslap - Load testing and benchmarking a server ================================================== - --------- SYNOPSIS -------- -memaslap [options] +|client_prefix|\aslap [options] .. program:: memaslap .. option:: --help +.. option:: --servers .. envvar:: MEMCACHED_SERVERS ------------ DESCRIPTION ----------- - :program:`memaslap` is a load generation and benchmark tool for memcached servers. It generates configurable workload such as threads, concurrency, connections, run time, overwrite, miss rate, key size, value size, get/set @@ -33,86 +30,52 @@ the threads don't communicate with each other, and there are several socket connections in each thread. Each connection keeps key size distribution, value size distribution, and command distribution by itself. -You can specify servers via the :option:`memslap --servers` option or via the +You can specify servers via the :option:`memaslap --servers` option or via the environment variable :envvar:`MEMCACHED_SERVERS`. - --------- FEATURES -------- - Memslap is developed to for the following purposes: - Manages network connections with libevent asynchronously. - - Set both TCP and UDP up to use non-blocking IO. - - Improves parallelism: higher performance in multi-threads environments. - - Improves time efficiency: faster processing speed. - - Generates key and value more efficiently; key size distribution and value size distribution are configurable. - - Supports get, multi-get, and set commands; command distribution is configurable. - - Supports controllable miss rate and overwrite rate. - - Supports data and expire-time verification. - - Supports dumping statistic information periodically. - - Supports thousands of TCP connections. - - Supports binary protocol. - - Supports facebook test (set with TCP and multi-get with UDP) and replication test. - - - -------- DETAILS ------- - Effective implementation of network. ____________________________________ - For memaslap, both TCP and UDP use non-blocking network IO. All the network events are managed by libevent as memcached. The network module of memaslap is similar to memcached. Libevent can ensure memaslap can handle network very efficiently. - Effective implementation of multi-threads and concurrency _________________________________________________________ - Memslap has the similar implementation of multi-threads to memcached. Memslap creates one or more self-governed threads; each thread is bound with one CPU core if the system tests setting CPU @@ -131,12 +94,10 @@ socket connection at any given time. Users can specify the number of concurrency and socket connections of each concurrency according to their expected workload. - Effective implementation of generating key and value ____________________________________________________ - -In order to improve time efficiency and space efficiency, +In order to improve time efficiency and space efficiency, memaslap creates a random characters table with 10M characters. All the suffixes of keys and values are generated from this random characters table. @@ -159,11 +120,9 @@ object from the window to do set operation or get operation. At the same time, each concurrency kicks objects out of its window and adds new object into it. - Simple but useful task scheduling _________________________________ - Memslap uses libevent to schedule all concurrent tasks of threads, and each concurrency schedules tasks based on the local task window. Memslap assumes that if each concurrency keeps the same @@ -191,11 +150,9 @@ cache layer. It causes the first cache layer to miss. So the user can specify the window size to get the expected miss rate of the first cache layer. - Useful implementation of multi-servers , UDP, TCP, multi-get and binary protocol ________________________________________________________________________________ - Because each thread is self-governed, memaslap can assign different threads to handle different memcached servers. This is just one of the ways in which memaslap tests multiple servers. The only @@ -219,44 +176,25 @@ one command from the server and reorders the response data. If some packages get lost, the waiting timeout mechanism can ensure half-baked packages will be discarded and the next command will be sent. - - ------ USAGE ----- - Below are some usage samples: - memaslap -s 127.0.0.1:11211 -S 5s - - memaslap -s 127.0.0.1:11211 -t 2m -v 0.2 -e 0.05 -b - - memaslap -s 127.0.0.1:11211 -F config -t 2m -w 40k -S 20s -o 0.2 - - memaslap -s 127.0.0.1:11211 -F config -t 2m -T 4 -c 128 -d 20 -P 40k - - memaslap -s 127.0.0.1:11211 -F config -t 2m -d 50 -a -n 40 - - memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m - - memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m -p 2 - - The user must specify one server at least to run memaslap. The rest of the parameters have default values, as shown below: @@ -284,11 +222,9 @@ TCP = true Limit throughput = false Facebook test = false Replication test = false - Key size, value size and command distribution. ______________________________________________ - All the distributions are read from the configuration file specified by user with "—cfg_cmd" option. If the user does not specify a configuration file, memaslap will run with the default distribution (key size = 64, @@ -306,11 +242,9 @@ Currently, memaslap only tests set and get commands. And it testss 100% set and 100% get. For 100% get, it will preset some objects to the server. - Multi-thread and concurrency ____________________________ - The high performance of memaslap benefits from the special schedule of thread and concurrency. It's important to specify the proper number of them. The default number of threads is 1; the default number of @@ -366,11 +300,9 @@ kops/s for a specific configuration, you can specify the throughput equal to or less than the maximum throughput using "--tps" option. - Window size ___________ - Most of the time, the user does not need to specify the window size. The default window size is 10k. For Schooner Memcached, the user can specify different window sizes to get different cache miss rates based on the test @@ -415,11 +347,9 @@ The formula for calculating window size for cache miss rate 5%: cache_size / concurrency / (key_size + value_size) \* 0.7 - Verification ____________ - Memslap testss both data verification and expire-time verification. The user can use "--verify=" or "-v" to specify the proportion of data verification. In theory, it testss 100% data verification. The @@ -428,16 +358,14 @@ expire-time verification. In theory, it testss 100% expire-time verification. Specify the "--verbose" options to get more detailed error information. -For example: --exp_verify=0.01 –verify=0.1 , it means that 1% of the objects +For example: --exp_verify=0.01 –verify=0.1 , it means that 1% of the objects set with expire-time, 10% of the objects gotten will be verified. If the objects are gotten, memaslap will verify the expire-time and value. - multi-servers and multi-config _______________________________ - Memslap testss multi-servers based on self-governed thread. There is a limitation that the number of servers cannot be greater than the number of threads. Memslap assigns one thread to handle one @@ -455,17 +383,15 @@ and 4 handle server 1 (10.1.1.2); and thread 2 and 5 handle server 2 All the threads and concurrencies in memaslap are self-governed. -So is memaslap. The user can start up several +So is memaslap. The user can start up several memaslap instances. The user can run memaslap on different client machines to communicate with the same memcached server at the same. It is recommended that the user start different memaslap on different machines using the same configuration. - Run with execute number mode or time mode _________________________________________ - The default memaslap runs with time mode. The default run time is 10 minutes. If it times out, memaslap will exit. Do not specify both execute number mode and time mode at the same time; just @@ -477,11 +403,9 @@ For example: --execute_number=100000 (It means that after running 100000 commands, the test will exit.) - Dump statistic information periodically. ________________________________________ - The user can use "--stat_freq=" or "-S" to specify the frequency. For example: @@ -493,11 +417,9 @@ seconds. For more information on the format of dumping statistic information, refer to "Format of Output" section. - Multi-get _________ - The user can use "--division=" or "-d" to specify multi-get keys count. Memslap by default does single get with TCP. Memslap also testss data verification and expire-time verification for multi-get. @@ -509,11 +431,9 @@ memaslap sends one "multi-get" to the server once. For the binary protocol, memaslap sends several single get commands together as "multi-get" to the server. - UDP and TCP ___________ - Memslap testss both UDP and TCP. For TCP, memaslap does not reconnect the memcached server if socket connections are lost. If all the socket connections are lost or memcached server crashes, @@ -530,11 +450,9 @@ memcached does not tests that. UDP doesn't tests reconnection. - Facebook test _____________ - Set data with TCP and multi-get with UDP. Specify the following options: "--facebook --division=50" @@ -554,11 +472,9 @@ objects once with the UDP socket. If you specify "--division=50", the key size must be less that 25 bytes because the UDP packet size is 1400 bytes. - Replication test ________________ - For replication test, the user must specify at least two memcached servers. The user can use "—rep_write=" option to enable feature. @@ -574,11 +490,9 @@ memaslap will only get objects from server 1. If server 0 comes back to life again, memaslap will reconnect server 0. If both server 0 and server 1 crash, memaslap will exit. - Supports thousands of TCP connections _____________________________________ - Start memaslap with "--conn_sock=" or "-n" to enable this feature. Make sure that your system can tests opening thousands of files and creating thousands of sockets. However, this feature does not tests @@ -593,11 +507,9 @@ thread has 16 concurrencies, each concurrency has 128 TCP socket connections, and the total number of TCP socket connections is 128 \* 128 = 16384. - Supports binary protocol ________________________ - Start memaslap with "--binary" or "-B" options to enable this feature. It testss all the above features except UDP, because the latest memcached 1.3.3 does not implement binary UDP protocol. @@ -611,20 +523,15 @@ memaslap does not tests UDP. In addition, memcached 1.3.3 does not tests multi-get. If you specify "--division=50" option, it just sends 50 get commands together as "multi-get" to the server. - - ------------------- Configuration file ------------------ - This section describes the format of the configuration file. By default when no configuration file is specified memaslap reads the default one located at ~/.memaslap.cnf. Below is a sample configuration file: - .. code-block:: perl --------------------------------------------------------------------------- @@ -699,94 +606,58 @@ Below is a sample configuration file: 0 0.1 1.0 0.9 - - ----------------- Format of output ---------------- - At the beginning, memaslap displays some configuration information as follows: - servers : 127.0.0.1:11211 - - threads count: 1 - - concurrency: 16 - - run time: 20s - - windows size: 10k - - set proportion: set_prop=0.10 - - get proportion: get_prop=0.90 - - Where _____ - - servers : "servers" The servers used by memaslap. - - threads count The number of threads memaslap runs with. - - concurrency The number of concurrencies memaslap runs with. - - run time How long to run memaslap. - - windows size The task window size of each concurrency. - - set proportion The proportion of set command. - - get proportion The proportion of get command. - - The output of dynamic statistics is something like this: - .. code-block:: perl --------------------------------------------------------------------------------------------------------------------------------- @@ -797,18 +668,16 @@ The output of dynamic statistics is something like this: 95.43 177.29 Global 20 1257935 62896 71.8 0 26 3791 224 117.79 192.60 - - Set Statistics + Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) Avg(us) Std_dev Geo_dist Period 5 38425 7685 7.3 0 42 628 240 88.05 220.21 Global 20 139780 6989 8.0 0 37 3790 253 117.93 224.83 - - Total Statistics + Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) Avg(us) Std_dev Geo_dist Period 5 384252 76850 72.5 0 27 2198 207 @@ -817,100 +686,67 @@ The output of dynamic statistics is something like this: 117.93 195.60 --------------------------------------------------------------------------------------------------------------------------------- - - Where _____ - - Get Statistics Statistics information of get command - - Set Statistics Statistics information of set command - - Total Statistics Statistics information of both get and set command - - Period Result within a period - - Global Accumulated results - - Ops Total operations - - TPS Throughput, operations/second - - Net The rate of network - - Get_miss How many objects can't be gotten - - Min The minimum response time - - Max The maximum response time - - Avg: The average response time - - Std_dev Standard deviation of response time - - Geo_dist Geometric distribution based on natural exponential function - - At the end, memaslap will output something like this: - .. code-block:: perl --------------------------------------------------------------------------------------------------------------------------------- @@ -964,170 +800,112 @@ At the end, memaslap will output something like this: Run time: 20.0s Ops: 1397754 TPS: 69817 Net_rate: 59.4M/s --------------------------------------------------------------------------------------------------------------------------------- - - Where _____ - - Get Statistics Get statistics of response time - - Set Statistics Set statistics of response time - - Total Statistics Both get and set statistics of response time - - Min The accumulated and minimum response time - - Max The accumulated and maximum response time - - Avg The accumulated and average response time - - Std Standard deviation of response time - - Log2 Dist Geometric distribution based on logarithm 2 - - cmd_get Total get commands done - - cmd_set Total set commands done - - get_misses How many objects can't be gotten from server - - verify_misses How many objects need to verify but can't get them - - verify_failed How many objects with insistent value - - expired_get How many objects are expired but we get them - - unexpired_unget How many objects are unexpired but we can't get them - - written_bytes Total written bytes - - read_bytes Total read bytes - - object_bytes Total object bytes - - packet_disorder How many UDP packages are disorder - - packet_drop How many UDP packages are lost - - udp_timeout How many times UDP time out happen - - Run time Total run time - - Ops Total operations - - TPS Throughput, operations/second - - Net_rate The average rate of network - - - - -------- OPTIONS ------- - -s, --servers= List one or more servers to connect. Servers count must be less than threads count. e.g.: --servers=localhost:1234,localhost:11211 @@ -1177,17 +955,17 @@ OPTIONS The proportion of objects need overwrite, e.g.: --overwrite=0.01. Default never overwrite object. --R, --reconnect +-R, --reconnect Reconnect tests, when connection is closed it will be reconnected. --U, --udp +-U, --udp UDP tests, default memaslap uses TCP, TCP port and UDP port of server must be same. --a, --facebook +-a, --facebook Whether it enables facebook test feature, set with TCP and multi-get with UDP. --B, --binary +-B, --binary Whether it enables binary protocol. Default with ASCII protocol. -P, --tps= @@ -1196,21 +974,18 @@ OPTIONS -p, --rep_write= The first nth servers can write data, e.g.: --rep_write=2. --b, --verbose +-b, --verbose Whether it outputs detailed information when verification fails. --h, --help +-h, --help Display this message and then exit. --V, --version +-V, --version Display the version of the application and then exit. - --------- EXAMPLES -------- - memaslap -s 127.0.0.1:11211 -S 5s memaslap -s 127.0.0.1:11211 -t 2m -v 0.2 -e 0.05 -b @@ -1225,8 +1000,15 @@ memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m -p 2 --------- +NOTES +----- + +.. include:: common/note_program_prefix.rst +.. include:: common/note_contrib_options.rst + SEE ALSO -------- -:manpage:`memcached(1)` :manpage:`libmemcached(3)` +.. only:: man + + :manpage:`memcached(1)` :manpage:`libmemcached(3)`