From: Brian Aker Date: Fri, 24 Jun 2011 19:25:01 +0000 (-0700) Subject: Update docs, and syncronize libtest. X-Git-Tag: 0.51~3^2 X-Git-Url: https://git.m6w6.name/?a=commitdiff_plain;h=ecc7382f48857cb22e74ffa9496f9a31e956bde3;p=awesomized%2Flibmemcached Update docs, and syncronize libtest. --- diff --git a/docs/bin/memaslap.rst b/docs/bin/memaslap.rst new file mode 100644 index 00000000..b69bb3f8 --- /dev/null +++ b/docs/bin/memaslap.rst @@ -0,0 +1,1248 @@ +================================================== +memaslap - Load testing and benchmarking a server +================================================== + + +-------- +SYNOPSIS +-------- + +memaslap [options] + +.. program:: memaslap + +.. option:: --help + +----------- +DESCRIPTION +----------- + + +memaslap is a load generation and benchmark tool for :program:`memcached()` +servers. It generates configurable workload such as threads, concurrencies, connections, run time, overwrite, miss rate, key size, value size, get/set proportion, expected throughput, and so on. Furthermore, it also testss data +verification, expire-time verification, UDP, binary protocol, facebook test, +replication test, multi-get and reconnection, etc. + +Memslap manages network connections like memcached with +libevent. Each thread of memaslap is bound with a CPU core, all +the threads don't communicate with each other, and there are several socket +connections in each thread. Each connection keeps key size distribution, +value size distribution, and command distribution by itself. + +You can specify servers via the :option:`--servers` option or via the +environment variable :envvar:`MEMCACHED_SERVERS`. + + +-------- +FEATURES +-------- + + +Memslap is developed to for the following purposes: + + +Manages network connections with libevent asynchronously. + + + +Set both TCP and UDP up to use non-blocking IO. + + + +Improves parallelism: higher performance in multi-threads environments. + + + +Improves time efficiency: faster processing speed. + + + +Generates key and value more efficiently; key size distribution and value size distribution are configurable. + + + +Supports get, multi-get, and set commands; command distribution is configurable. + + + +Supports controllable miss rate and overwrite rate. + + + +Supports data and expire-time verification. + + + +Supports dumping statistic information periodically. + + + +Supports thousands of TCP connections. + + + +Supports binary protocol. + + + +Supports facebook test (set with TCP and multi-get with UDP) and replication test. + + + + +------- +DETAILS +------- + + +Effective implementation of network. +____________________________________ + + +For memaslap, both TCP and UDP use non-blocking network IO. All +the network events are managed by libevent as memcached. The network module +of memaslap is similar to memcached. Libevent can ensure +memaslap can handle network very efficiently. + + +Effective implementation of multi-threads and concurrency +_________________________________________________________ + + +Memslap has the similar implementation of multi-threads to +memcached. Memslap creates one or more self-governed threads; +each thread is bound with one CPU core if the system testss setting CPU +core affinity. + +In addition, each thread has a libevent to manage the events of the network; +each thread has one or more self-governed concurrencies; and each +concurrency has one or more socket connections. All the concurrencies don’t +communicate with each other even though they are in the same thread. + +Memslap can create thousands of socket connections, and each +concurrency has tens of socket connections. Each concurrency randomly or +sequentially selects one socket connection from its socket connection pool +to run, so memaslap can ensure each concurrency handles one +socket connection at any given time. Users can specify the number of +concurrency and socket connections of each concurrency according to their +expected workload. + + +Effective implementation of generating key and value +____________________________________________________ + + +In order to improve time efficiency and space efficiency, +memaslap creates a random characters table with 10M characters. All the +suffixes of keys and values are generated from this random characters table. + +Memslap uses the offset in the character table and the length +of the string to identify a string. It can save much memory. +Each key contains two parts, a prefix and a suffix. The prefix is an +uint64_t, 8 bytes. In order to verify the data set before, +memaslap need to ensure each key is unique, so it uses the prefix to identify +a key. The prefix cannot include illegal characters, such as ‘\r’, ‘\n’, +‘\0’ and ‘ ‘. And memaslap has an algorithm to ensure that. + +Memslap doesn’t generate all the objects (key-value pairs) at +the beginning. It only generates enough objects to fill the task window +(default 10K objects) of each concurrency. Each object has the following +basic information, key prefix, key suffix offset in the character table, key +length, value offset in the character table, and value length. + +In the work process, each concurrency sequentially or randomly selects an +object from the window to do set operation or get operation. At the same +time, each concurrency kicks objects out of its window and adds new object +into it. + + +Simple but useful task scheduling +_________________________________ + + +Memslap uses libevent to schedule all the concurrencies of +threads, and each concurrency schedules tasks based on the local task +window. Memslap assumes that if each concurrency keeps the same +key distribution, value distribution and commands distribution, from +outside, memaslap keeps all the distribution as a whole. +Each task window includes a lot of objects, each object stores its basic +information, such as key, value, expire time, and so on. At any time, all +the objects in the window keep the same and fixed key and value +distribution. If an object is overwritten, the value of the object will be +updated. Memslap verifies the data or expire-time according to +the object information stored in the task window. + +Libevent selects which concurrency to handle based on a specific network +event. Then the concurrency selects which command (get or set) to operate +based on the command distribution. If it needs to kick out an old object and +add a new object, in order to keep the same key and value distribution, the +new object must have the same key length and value length. + +If memcached server has two cache layers (memory and SSD), running +memaslap with different window sizes can get different cache +miss rates. If memaslap adds enough objects into the windows at +the beginning, and the cache of memcached cannot store all the objects +initialized, then memaslap will get some objects from the second +cache layer. It causes the first cache layer to miss. So the user can +specify the window size to get the expected miss rate of the first cache +layer. + + +Useful implementation of multi-servers , UDP, TCP, multi-get and binary protocol +________________________________________________________________________________ + + +Because each thread is self-governed, memaslap can assign +different threads to handle different memcached servers. This is just one of +the ways in which memaslap testss multiple servers. The only +limitation is that the number of servers cannot be greater than the number +of threads. The other way to tests multiple servers is for replication +test. Each concurrency has one socket connection to each memcached server. +For the implementation, memaslap can set some objects to one +memcached server, and get these objects from the other servers. + +By default, Memslap does single get. If the user specifies +multi-get option, memaslap will collect enough get commands and +pack and send the commands together. + +Memslap testss both the ASCII protocol and binary protocol, +but it runs on the ASCII protocol by default. +Memslap by default runs on the TCP protocol, but it also +testss UDP. Because UDP is unreliable, dropped packages and out-of-order +packages may occur. Memslap creates a memory buffer to handle +these problems. Memslap tries to read all the response data of +one command from the server and reorders the response data. If some packages +get lost, the waiting timeout mechanism can ensure half-baked packages will +be discarded and the next command will be sent. + + + +----- +USAGE +----- + + +Below are some usage samples: + + +memaslap -s 127.0.0.1:11211 -S 5s + + + +memaslap -s 127.0.0.1:11211 -t 2m -v 0.2 -e 0.05 -b + + + +memaslap -s 127.0.0.1:11211 -F config -t 2m -w 40k -S 20s -o 0.2 + + + +memaslap -s 127.0.0.1:11211 -F config -t 2m -T 4 -c 128 -d 20 -P 40k + + + +memaslap -s 127.0.0.1:11211 -F config -t 2m -d 50 -a -n 40 + + + +memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m + + + +memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m -p 2 + + + +The user must specify one server at least to run memaslap. The +rest of the parameters have default values, as shown below: + +Thread number = 1 Concurrency = 16 + +Run time = 600 seconds Configuration file = NULL + +Key size = 64 Value size = 1024 + +Get/set = 9:1 Window size = 10k + +Execute number = 0 Single get = true + +Multi-get = false Number of sockets of each concurrency = 1 + +Reconnect = false Data verification = false + +Expire-time verification = false ASCII protocol = true + +Binary protocol = false Dumping statistic information + +periodically = false + +Overwrite proportion = 0% UDP = false + +TCP = true Limit throughput = false + +Facebook test = false Replication test = false + +Key size, value size and command distribution. +______________________________________________ + + +All the distributions are read from the configuration file specified by user +with “—cfg_cmd” option. If the user does not specify a configuration file, +memaslap will run with the default distribution (key size = 64, +value size = 1024, get/set = 9:1). For information on how to edit the +configuration file, refer to the “Configuration File” section. + +The minimum key size is 16 bytes; the maximum key size is 250 bytes. The +precision of proportion is 0.001. The proportion of distribution will be +rounded to 3 decimal places. + +The minimum value size is 1 bytes; the maximum value size is 1M bytes. The +precision of proportion is 0.001. The proportion of distribution will be +rounded to 3 decimal places. +Currently, memaslap only testss set and get commands. And it +testss 100% set and 100% get. For 100% get, it will preset some objects to +the server. + + +Multi-thread and concurrency +____________________________ + + +The high performance of memaslap benefits from the special +schedule of thread and concurrency. It’s important to specify the proper +number of them. The default number of threads is 1; the default number of +concurrency is 16. The user can use “—threads” and “--concurrency” to +specify these variables. + +If the system testss setting CPU affinity and the number of threads +specified by the user is greater than 1, memaslap will try to +bind each thread to a different CPU core. So if you want to get the best +performance memaslap, it is better to specify the number of +thread equal to the number of CPU cores. The number of threads specified by +the user can also be less or greater than the number of CPU cores. Because +of the limitation of implementation, the number of concurrencies could be +the multiple of the number of threads. + +1. For 8 CPU cores system + +For example: + +--threads=2 --concurrency=128 + +--threads=8 --concurrency=128 + +--threads=8 --concurrency=256 + +--threads=12 --concurrency=144 + +2. For 16 CPU cores system + +For example: + +--threads=8 --concurrency=128 + +--threads=16 --concurrency=256 + +--threads=16 --concurrency=512 + +--threads=24 --concurrency=288 + +The memaslap performs very well, when +used to test the performance of memcached servers. +Most of the time, the bottleneck is the network or +the server. If for some reason the user wants to +limit the performance of memaslap, there +are two ways to do this: + +Decrease the number of threads and concurrencies. +Use the option “--tps” that memaslap +provides to limit the throughput. This option allows +the user to get the expected throughput. For +example, assume that the maximum throughput is 50 +kops/s for a specific configuration, you can specify +the throughput equal to or less than the maximum +throughput using “--tps” option. + + +Window size +___________ + + +Most of the time, the user does not need to specify the window size. The +default window size is 10k. For Schooner Memcached, the user can specify +different window sizes to get different cache miss rates based on the test +case. Memslap testss cache miss rate between 0% and 100%. +If you use this utility to test the performance of Schooner Memcached, you +can specify a proper window size to get the expected cache miss rate. The +formula for calculating window size is as follows: + +Assume that the key size is 128 bytes, and the value size is 2048 bytes, and +concurrency=128. + +1. Small cache cache_size=1M, 100% cache miss (all data get from SSD). +win_size=10k + +2. cache_size=4G + +(1). cache miss rate 0% + +win_size=8k + +(2). cache miss rate 5% + +win_size=11k + +3. cache_size=16G + +(1). cache miss rate 0% + +win_size=32k + +(2). cache miss + +rate 5% + +win_size=46k + +The formula for calculating window size for cache miss rate 0%: + +cache_size / concurrency / (key_size + value_size) \* 0.5 + +The formula for calculating window size for cache miss rate 5%: + +cache_size / concurrency / (key_size + value_size) \* 0.7 + + +Verification +____________ + + +Memslap testss both data verification and expire-time +verification. The user can use "--verify=" or "-v" to specify the proportion +of data verification. In theory, it testss 100% data verification. The +user can use "--exp_verify=" or "-e" to specify the proportion of +expire-time verification. In theory, it testss 100% expire-time +verification. Specify the "--verbose" options to get more detailed error +information. + +For example: --exp_verify=0.01 –verify=0.1 , it means that 1% of the objects +set with expire-time, 10% of the objects gotten will be verified. If the +objects are gotten, memaslap will verify the expire-time and +value. + + +multi-servers and multi-config +_______________________________ + + +Memslap testss multi-servers based on self-governed thread. +There is a limitation that the number of servers cannot be greater than the +number of threads. Memslap assigns one thread to handle one +server at least. The user can use the "--servers=" or "-s" option to specify +multi-servers. + +For example: + +--servers=10.1.1.1:11211,10.1.1.2:11212,10.1.1.3:11213 --threads=6 --concurrency=36 + +The above command means that there are 6 threads, with each thread having 6 +concurrencies and that threads 0 and 3 handle server 0 (10.1.1.1); threads 1 +and 4 handle server 1 (10.1.1.2); and thread 2 and 5 handle server 2 +(10.1.1.3). + +All the threads and concurrencies in memaslap are self-governed. + +So is memaslap. The user can start up several +memaslap instances. The user can run memaslap on different client +machines to communicate with the same memcached server at the same. It is +recommended that the user start different memaslap on different +machines using the same configuration. + + +Run with execute number mode or time mode +_________________________________________ + + +The default memaslap runs with time mode. The default run time +is 10 minutes. If it times out, memaslap will exit. Do not +specify both execute number mode and time mode at the same time; just +specify one instead. + +For example: + +--time=30s (It means the test will run 30 seconds.) + +--execute_number=100000 (It means that after running 100000 commands, the test will exit.) + + +Dump statistic information periodically. +________________________________________ + + +The user can use "--stat_freq=" or "-S" to specify the frequency. + +For example: + +--stat_freq=20s + +Memslap will dump the statistics of the commands (get and set) at the frequency of every 20 +seconds. + +For more information on the format of dumping statistic information, refer to “Format of Output” section. + + +Multi-get +_________ + + +The user can use "--division=" or "-d" to specify multi-get keys count. +Memslap by default does single get with TCP. Memslap also testss data +verification and expire-time verification for multi-get. + +Memslap testss multi-get with both TCP and UDP. Because of +the different implementation of the ASCII protocol and binary protocol, +there are some differences between the two. For the ASCII protocol, +memaslap sends one “multi-get” to the server once. For the +binary protocol, memaslap sends several single get commands +together as “multi-get” to the server. + + +UDP and TCP +___________ + + +Memslap testss both UDP and TCP. For TCP, +memaslap does not reconnect the memcached server if socket connections are +lost. If all the socket connections are lost or memcached server crashes, +memaslap will exit. If the user specifies the “--reconnect” +option when socket connections are lost, it will reconnect them. + +User can use “--udp” to enable the UDP feature, but UDP comes with some +limitations: + +UDP cannot set data more than 1400 bytes. + +UDP is not testsed by the binary protocol because the binary protocol of +memcached does not tests that. + +UDP doesn’t tests reconnection. + + +Facebook test +_____________ + + +Set data with TCP and multi-get with UDP. Specify the following options: + +"--facebook --division=50" + +If you want to create thousands of TCP connections, specify the + +"--conn_sock=" option. + +For example: --facebook --division=50 --conn_sock=200 + +The above command means that memaslap will do facebook test, +each concurrency has 200 socket TCP connections and one UDP socket. + +Memslap sets objects with the TCP socket, and multi-gets 50 +objects once with the UDP socket. + +If you specify "--division=50", the key size must be less that 25 bytes +because the UDP packet size is 1400 bytes. + + +Replication test +________________ + + +For replication test, the user must specify at least two memcached servers. +The user can use “—rep_write=” option to enable feature. + +For example: + +--servers=10.1.1.1:11211,10.1.1.2:11212 –rep_write=2 + +The above command means that there are 2 replication memcached servers, +memaslap will set objects to both server 0 and server 1, get +objects which are set to server 0 before from server 1, and also get objects +which are set to server 1 before from server 0. If server 0 crashes, +memaslap will only get objects from server 1. If server 0 comes +back to life again, memaslap will reconnect server 0. If both +server 0 and server 1 crash, memaslap will exit. + + +Supports thousands of TCP connections +_____________________________________ + + +Start memaslap with "--conn_sock=" or "-n" to enable this +feature. Make sure that your system can tests opening thousands of files +and creating thousands of sockets. However, this feature does not tests +reconnection if sockets disconnect. + +For example: + +--threads=8 --concurrency=128 --conn_sock=128 + +The above command means that memaslap starts up 8 threads, each +thread has 16 concurrencies, each concurrency has 128 TCP socket +connections, and the total number of TCP socket connections is 128 \* 128 = +16384. + + +Supports binary protocol +________________________ + + +Start memaslap with "--binary" or "-B" options to enable this +feature. It testss all the above features except UDP, because the latest +memcached 1.3.3 does not implement binary UDP protocol. + +For example: + +--binary + +Since memcached 1.3.3 doesn't implement binary UDP protocol, +memaslap does not tests UDP. In addition, memcached 1.3.3 does not tests +multi-get. If you specify "--division=50" option, it just sends 50 get +commands together as “mulit-get” to the server. + + + +------------------ +Configuration file +------------------ + + +This section describes the format of the configuration file. By default +when no configuration file is specified memaslap reads the default +one located at ~/.memaslap.cnf. + +Below is a sample configuration file: + + +.. code-block:: perl + + --------------------------------------------------------------------------- + #comments should start with '#' + #key + #start_len end_len proportion + # + #key length range from start_len to end_len + #start_len must be equal to or greater than 16 + #end_len must be equal to or less than 250 + #start_len must be equal to or greater than end_len + #memaslap will generate keys according to the key range + #proportion: indicates keys generated from one range accounts for the total + generated keys + # + #example1: key range 16~100 accounts for 80% + # key range 101~200 accounts for 10% + # key range 201~250 accounts for 10% + # total should be 1 (0.8+0.1+0.1 = 1) + # + # 16 100 0.8 + # 101 200 0.1 + # 201 249 0.1 + # + #example2: all keys length are 128 bytes + # + # 128 128 1 + key + 128 128 1 + #value + #start_len end_len proportion + # + #value length range from start_len to end_len + #start_len must be equal to or greater than 1 + #end_len must be equal to or less than 1M + #start_len must be equal to or greater than end_len + #memaslap will generate values according to the value range + #proportion: indicates values generated from one range accounts for the + total generated values + # + #example1: value range 1~1000 accounts for 80% + # value range 1001~10000 accounts for 10% + # value range 10001~100000 accounts for 10% + # total should be 1 (0.8+0.1+0.1 = 1) + # + # 1 1000 0.8 + # 1001 10000 0.1 + # 10001 100000 0.1 + # + #example2: all value length are 128 bytes + # + # 128 128 1 + value + 2048 2048 1 + #cmd + #cmd_type cmd_proportion + # + #currently memaslap only testss get and set command. + # + #cmd_type + #set 0 + #get 1 + # + #example: set command accounts for 50% + # get command accounts for 50% + # total should be 1 (0.5+0.5 = 1) + # + # cmd + # 0 0.5 + # 1 0.5 + cmd + 0 0.1 + 1.0 0.9 + + + +---------------- +Format of output +---------------- + + +At the beginning, memaslap displays some configuration information as follows: + + +servers : 127.0.0.1:11211 + + + +threads count: 1 + + + +concurrency: 16 + + + +run time: 20s + + + +windows size: 10k + + + +set proportion: set_prop=0.10 + + + +get proportion: get_prop=0.90 + + + +Where +_____ + + + +servers : "servers" + + The servers used by memaslap. + + + +threads count + + The number of threads memaslap runs with. + + + +concurrency + + The number of concurrencies memaslap runs with. + + + +run time + + How long to run memaslap. + + + +windows size + + The task window size of each concurrency. + + + +set proportion + + The proportion of set command. + + + +get proportion + + The proportion of get command. + + + +The output of dynamic statistics is something like this: + + +.. code-block:: perl + + --------------------------------------------------------------------------------------------------------------------------------- + Get Statistics + Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) + Avg(us) Std_dev Geo_dist + Period 5 345826 69165 65.3 0 27 2198 203 + 95.43 177.29 + Global 20 1257935 62896 71.8 0 26 3791 224 + 117.79 192.60 + + + Set Statistics + Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) + Avg(us) Std_dev Geo_dist + Period 5 38425 7685 7.3 0 42 628 240 + 88.05 220.21 + Global 20 139780 6989 8.0 0 37 3790 253 + 117.93 224.83 + + + Total Statistics + Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) + Avg(us) Std_dev Geo_dist + Period 5 384252 76850 72.5 0 27 2198 207 + 94.72 181.18 + Global 20 1397720 69886 79.7 0 26 3791 227 + 117.93 195.60 + --------------------------------------------------------------------------------------------------------------------------------- + + + +Where +_____ + + + +Get Statistics + + Statistics information of get command + + + +Set Statistics + + Statistics information of set command + + + +Total Statistics + + Statistics information of both get and set command + + + +Period + + Result within a period + + + +Global + + Accumulated results + + + +Ops + + Total operations + + + +TPS + + Throughput, operations/second + + + +Net + + The rate of network + + + +Get_miss + + How many objects can’t be gotten + + + +Min + + The minimum response time + + + +Max + + The maximum response time + + + +Avg: + + The average response time + + + +Std_dev + + Standard deviation of response time + + + +Geo_dist + + Geometric distribution based on natural exponential function + + + +At the end, memaslap will output something like this: + + +.. code-block:: perl + + --------------------------------------------------------------------------------------------------------------------------------- + Get Statistics (1257956 events) + Min: 26 + Max: 3791 + Avg: 224 + Geo: 192.60 + Std: 116.23 + Log2 Dist: + 4: 0 10 84490 215345 + 8: 484890 459823 12543 824 + 12: 31 + + Set Statistics (139782 events) + Min: 37 + Max: 3790 + Avg: 253 + Geo: 224.84 + Std: 116.83 + Log2 Dist: + 4: 0 0 4200 16988 + 8: 50784 65574 2064 167 + 12: 5 + + Total Statistics (1397738 events) + Min: 26 + Max: 3791 + Avg: 227 + Geo: 195.60 + Std: 116.60 + Log2 Dist: + 4: 0 10 88690 232333 + 8: 535674 525397 14607 991 + 12: 36 + + cmd_get: 1257969 + cmd_set: 139785 + get_misses: 0 + verify_misses: 0 + verify_failed: 0 + expired_get: 0 + unexpired_unget: 0 + written_bytes: 242516030 + read_bytes: 1003702556 + object_bytes: 152086080 + packet_disorder: 0 + packet_drop: 0 + udp_timeout: 0 + + Run time: 20.0s Ops: 1397754 TPS: 69817 Net_rate: 59.4M/s + --------------------------------------------------------------------------------------------------------------------------------- + + + +Where +_____ + + + +Get Statistics + + Get statistics of response time + + + +Set Statistics + + Set statistics of response time + + + +Total Statistics + + Both get and set statistics of response time + + + +Min + + The accumulated and minimum response time + + + +Max + + The accumulated and maximum response time + + + +Avg + + The accumulated and average response time + + + +Std + + Standard deviation of response time + + + +Log2 Dist + + Geometric distribution based on logarithm 2 + + + +cmd_get + + Total get commands done + + + +cmd_set + + Total set commands done + + + +get_misses + + How many objects can’t be gotten from server + + + +verify_misses + + How many objects need to verify but can’t get them + + + +verify_failed + + How many objects with insistent value + + + +expired_get + + How many objects are expired but we get them + + + +unexpired_unget + + How many objects are unexpired but we can’t get them + + + +written_bytes + + Total written bytes + + + +read_bytes + + Total read bytes + + + +object_bytes + + Total object bytes + + + +packet_disorder + + How many UDP packages are disorder + + + +packet_drop + + How many UDP packages are lost + + + +udp_timeout + + How many times UDP time out happen + + + +Run time + + Total run time + + + +Ops + + Total operations + + + +TPS + + Throughput, operations/second + + + +Net_rate + + The average rate of network + + + + + +------- +OPTIONS +------- + + +-s, --servers= + List one or more servers to connect. Servers count must be less than + threads count. e.g.: --servers=localhost:1234,localhost:11211 + +-T, --threads= + Number of threads to startup, better equal to CPU numbers. Default 8. + +-c, --concurrency= + Number of concurrency to simulate with load. Default 128. + +-n, --conn_sock= + Number of TCP socks per concurrency. Default 1. + +-x, --execute_number= + Number of operations(get and set) to execute for the + given test. Default 1000000. + +-t, --time= + How long the test to run, suffix: s-seconds, m-minutes, h-hours, + d-days e.g.: --time=2h. + +-F, --cfg_cmd= + Load the configure file to get command,key and value distribution list. + +-w, --win_size= + Task window size of each concurrency, suffix: K, M e.g.: --win_size=10k. + Default 10k. + +-X, --fixed_size= + Fixed length of value. + +-v, --verify= + The proportion of date verification, e.g.: --verify=0.01 + +-d, --division= + Number of keys to multi-get once. Default 1, means single get. + +-S, --stat_freq= + Frequency of dumping statistic information. suffix: s-seconds, + m-minutes, e.g.: --resp_freq=10s. + +-e, --exp_verify= + The proportion of objects with expire time, e.g.: --exp_verify=0.01. + Default no object with expire time + +-o, --overwrite= + The proportion of objects need overwrite, e.g.: --overwrite=0.01. + Default never overwrite object. + +-R, --reconnect + Reconnect tests, when connection is closed it will be reconnected. + +-U, --udp + UDP tests, default memaslap uses TCP, TCP port and UDP port of + server must be same. + +-a, --facebook + Whether it enables facebook test feature, set with TCP and multi-get with UDP. + +-B, --binary + Whether it enables binary protocol. Default with ASCII protocol. + +-P, --tps= + Expected throughput, suffix: K, e.g.: --tps=10k. + +-p, --rep_write= + The first nth servers can write data, e.g.: --rep_write=2. + +-b, --verbose + Whether it outputs detailed information when verification fails. + +-h, --help + Display this message and then exit. + +-V, --version + Display the version of the application and then exit. + + +-------- +EXAMPLES +-------- + + +memaslap -s 127.0.0.1:11211 -S 5s + +memaslap -s 127.0.0.1:11211 -t 2m -v 0.2 -e 0.05 -b + +memaslap -s 127.0.0.1:11211 -F config -t 2m -w 40k -S 20s -o 0.2 + +memaslap -s 127.0.0.1:11211 -F config -t 2m -T 4 -c 128 -d 20 -P 40k + +memaslap -s 127.0.0.1:11211 -F config -t 2m -d 50 -a -n 40 + +memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m + +memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m -p 2 + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +------- +AUTHORS +------- + + +Mingqiang Zhuang (Schooner Technolgy) +Brian Aker, + + +-------- +SEE ALSO +-------- + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/bin/memcapable.rst b/docs/bin/memcapable.rst new file mode 100644 index 00000000..0e4f33e3 --- /dev/null +++ b/docs/bin/memcapable.rst @@ -0,0 +1,78 @@ +======================================================================= +memcapable - Checking a Memcached server capibilities and compatibility +======================================================================= + +-------- +SYNOPSIS +-------- + +memcapable [options] + +.. program:: memcapable + +----------- +DESCRIPTION +----------- + +:program:`memcapable` connects to the specified memcached server and tries to +determine its capabilities by running the various commands and verifying +the response. + + +----------- +LIMITATIONS +----------- + + +The current version of memcapable will only verify the binary protocol. + + +------- +OPTIONS +------- + + +.. option:: -h hostname + +Specify the hostname to connect to. The default is \ *localhost*\ + +.. option:: -p port + +Specify the port number to connect to. The default is \ *11211*\ + +.. option:: -c + +Generate a coredump when it detects an error from the server. + +.. option:: -v + +Print out the comparison when it detects an error from the server. + +.. option:: -t n + +Set the timeout from an IO operation to/from the server to \ *n*\ seconds. + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +------ +AUTHOR +------ + + +Trond Norbye, + + +-------- +SEE ALSO +-------- + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` + diff --git a/docs/bin/memcat.rst b/docs/bin/memcat.rst new file mode 100644 index 00000000..b8612a92 --- /dev/null +++ b/docs/bin/memcat.rst @@ -0,0 +1,73 @@ +================================= +memcat - "cat" data from a server +================================= + + + +-------- +SYNOPSIS +-------- + +memcat [options] key + +Copy a set of keys to stdout + + +.. program:: memcat + + +----------- +DESCRIPTION +----------- + + +:program:`memcat` outputs to stdout the value a single or multiple set of keys +stored in a memcached(1) server. If any key is not found an error is returned. + +It is similar to the standard UNIX cat(1) utility. + + +------- +OPTIONS +------- + + +You can specify servers via the option: + +.. cmdoption:: --servers + +or via the environment variable: + +.. envvar:: `MEMCACHED_SERVERS` + +For a full list of operations run the tool with the option: + +.. cmdoption:: --help + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +------ +AUTHOR +------ + + +Brian Aker, + +Mark Atwood + + +-------- +SEE ALSO +-------- + + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` + diff --git a/docs/bin/memcp.rst b/docs/bin/memcp.rst new file mode 100644 index 00000000..a85eed8b --- /dev/null +++ b/docs/bin/memcp.rst @@ -0,0 +1,75 @@ +============================= +memcp - Copy data to a server +============================= + + +Copies files to a collection of memcached servers + + +-------- +SYNOPSIS +-------- + +memcp [options] [file] [server] + +.. program:: memcp + + +----------- +DESCRIPTION +----------- + + +:program:`memcp` copies one or more files into memcached(1) servers. +It is similar to the standard UNIX cp(1) command. + +The key names will be the names of the files, +without any directory path. + + +------- +OPTIONS +------- + + +You can specify servers via the option: + +.. option:: --servers + +or via the environment variable: + +.. envvar:: `MEMCACHED_SERVERS` + +If you do not specify either these, the final value in the command line list is the name of a server(s). + +For a full list of operations run the tool with the option: + +.. option:: --help + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +------ +AUTHOR +------ + + +Brian Aker, + +Mark Atwood, + + +-------- +SEE ALSO +-------- + + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` + diff --git a/docs/bin/memdump.rst b/docs/bin/memdump.rst new file mode 100644 index 00000000..6a8e12da --- /dev/null +++ b/docs/bin/memdump.rst @@ -0,0 +1,58 @@ +============================= +memdump - Dumping your server +============================= + + +Dump a list of keys from a server. + + +-------- +SYNOPSIS +-------- + +memdump [options] + +.. program:: memdump + +----------- +DESCRIPTION +----------- + + +:program:`memdump` dumps a list of "keys" from all servers that +it is told to fetch from. Because memcached does not guarentee to +provide all keys it is not possible to get a complete "dump". + + +------- +OPTIONS +------- + + +For a full list of operations run the tool with option: + +.. option:: --help + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +------ +AUTHOR +------ + + +Brian Aker, + + +-------- +SEE ALSO +-------- + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/bin/memerror.rst b/docs/bin/memerror.rst new file mode 100644 index 00000000..53fc09c4 --- /dev/null +++ b/docs/bin/memerror.rst @@ -0,0 +1,58 @@ +============================================== +memerror - translate an error code to a string +============================================== + + +Translates a memcached error code into a string + + +-------- +SYNOPSIS +-------- + +memerror [options] [error code] + +.. program:: memerror + +----------- +DESCRIPTION +----------- + + +:program:`memerror` translates an error code from libmemcached(3) into a human +readable string. + + +------- +OPTIONS +------- + + +For a full list of operations run the tool with option: + +.. option:: --help + + +---- +HOME +---- + + +To find out more infoerroration please check: +`http://libmemcached.org/ `_ + + +------ +AUTHOR +------ + + +Brian Aker, + + +-------- +SEE ALSO +-------- + + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/bin/memflush.rst b/docs/bin/memflush.rst new file mode 100644 index 00000000..a2b9894d --- /dev/null +++ b/docs/bin/memflush.rst @@ -0,0 +1,67 @@ +======================================= +memflush - flush all data from a server +======================================= + + +Reset a server or list of servers + + +-------- +SYNOPSIS +-------- + +memflush [options] + +.. program:: memflush + +----------- +DESCRIPTION +----------- + + +:program:`memflush` resets the contents of memcached(1) servers. +This means that all data in the specified servers will be deleted. + + +------- +OPTIONS +------- + + +You can specify servers via the option: + +.. option:: --servers + +or via the environment variable: + +.. envvar:: `MEMCACHED_SERVERS` + +For a full list of operations run the tool with option: + +.. option:: --help + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +------ +AUTHOR +------ + + +Brian Aker, + +Mark Atwood + + +-------- +SEE ALSO +-------- + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/bin/memrm.rst b/docs/bin/memrm.rst new file mode 100644 index 00000000..d395b364 --- /dev/null +++ b/docs/bin/memrm.rst @@ -0,0 +1,71 @@ +================================= +memrm - Remove data from a server +================================= + + +memrm - Remove a key(s) from a collection of memcached servers + + +-------- +SYNOPSIS +-------- + +memrm [options] [key] + +.. program:: memrm + + +----------- +DESCRIPTION +----------- + + +:program:`memrm` removes items, specified by key, from memcached(1) servers. + + +------- +OPTIONS +------- + + +You can specify servers via the option: + +.. option:: --servers + +or via the environment variable: + +.. envvar:: `MEMCACHED_SERVERS` + +For a full list of operations run the tool with the + +.. option:: --help + + + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +------ +AUTHOR +------ + + +Brian Aker, + +Mark Atwood, + + +-------- +SEE ALSO +-------- + + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` + diff --git a/docs/bin/memslap.rst b/docs/bin/memslap.rst new file mode 100644 index 00000000..fbaa5fe3 --- /dev/null +++ b/docs/bin/memslap.rst @@ -0,0 +1,63 @@ +================================================= +memslap - Load testing and benchmarking a server +================================================= + + +-------- +SYNOPSIS +-------- + +memslap [options] + +.. program:: memslap + +----------- +DESCRIPTION +----------- + + +:program:`memslap` is a load generation and benchmark tool for memcached(1) +servers. It generates configurable workload such as threads, concurrencies, connections, run time, overwrite, miss rate, key size, value size, get/set proportion, expected throughput, and so on. + + +------- +OPTIONS +------- + + +You can specify servers via the option: + +.. option:: --servers + +or via the environment variable: + +.. envvar:: `MEMCACHED_SERVERS` + +For a full list of operations run the tool with: + +.. option:: --help + + +---- +HOME +---- + + +To find out more information please check: +'http://libmemcached.org/ '_ + + +------ +AUTHOR +------ + + +Brian Aker, + + +-------- +SEE ALSO +-------- + + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/bin/memstat.rst b/docs/bin/memstat.rst new file mode 100644 index 00000000..3bbec8f8 --- /dev/null +++ b/docs/bin/memstat.rst @@ -0,0 +1,61 @@ +========================================= +memstat - Gather statistics from a server +========================================= + + +memstat - Display the operating status of a single or group of memcached servers + + +-------- +SYNOPSIS +-------- + +memstat [options] + +.. program:: memstat + +----------- +DESCRIPTION +----------- + + +:program:`memstat` dumps the state of memcached(1) servers. +It displays all data to stdout. + + +------- +OPTIONS +------- + + +You can specify servers via the option: + +.. option:: --servers + +or via the environment variable: + +.. envvar:: 'MEMCACHED_SERVERS, --args' + +which can be used to specify the "argument" sent to the stats command (ie slab, size, items, etc..). + +For a full list of operations run the tool with: + +.. option:: --help + + +---- +HOME +---- + + +To find out more information please check: +`http://libmemcached.org/ `_ + + +-------- +SEE ALSO +-------- + + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` + diff --git a/docs/conf.py.in b/docs/conf.py.in index a6a1dfb7..44f3f420 100644 --- a/docs/conf.py.in +++ b/docs/conf.py.in @@ -269,7 +269,7 @@ man_pages = [ ('memcached_flush_buffers', 'memcached_flush_buffers', u'libmemcached Documentation', [u'Brian Aker'], 3), ('memcached_generate_hash_value', 'memcached_generate_hash', u'Generating hash values directly', [u'Brian Aker'], 3), ('memcached_generate_hash_value', 'memcached_generate_hash_value', u'Generating hash values directly', [u'Brian Aker'], 3), - ('memcached_get', 'memcached_fetch', u'Retrieving data from the server', [u'Brian Aker'], 3), + ('libmemcached/memcached_fetch', 'memcached_fetch', u'Retrieving data from the server', [u'Brian Aker'], 3), ('memcached_get', 'memcached_fetch_execute', u'Retrieving data from the server', [u'Brian Aker'], 3), ('memcached_get', 'memcached_fetch_result', u'Retrieving data from the server', [u'Brian Aker'], 3), ('memcached_get', 'memcached_get', u'Retrieving data from the server', [u'Brian Aker'], 3), @@ -338,18 +338,13 @@ man_pages = [ ('memcached_verbosity', 'memcached_verbosity', u'libmemcached Documentation', [u'Brian Aker'], 3), ('memcached_version', 'memcached_lib_version', u'libmemcached Documentation', [u'Brian Aker'], 3), ('memcached_version', 'memcached_version', u'libmemcached Documentation', [u'Brian Aker'], 3), - ('memcapable', 'memcapable', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memcapable', 'memcapable', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memcat', 'memcat', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memcat', 'memcat', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memcp', 'memcp', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memcp', 'memcp', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memdump', 'memdump', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memdump', 'memdump', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memerror', 'memerror', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memerror', 'memerror', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memflush', 'memflush', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memrm', 'memrm', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memslap', 'memslap', u'libmemcached Documentation', [u'Brian Aker'], 1), - ('memstat', 'memstat', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memcapable', 'memcapable', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memcat', 'memcat', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memcp', 'memcp', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memdump', 'memdump', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memerror', 'memerror', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memflush', 'memflush', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memrm', 'memrm', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memslap', 'memslap', u'libmemcached Documentation', [u'Brian Aker'], 1), + ('bin/memstat', 'memstat', u'libmemcached Documentation', [u'Brian Aker'], 1), ] diff --git a/docs/index.rst b/docs/index.rst index 77b50600..57e2671d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -62,6 +62,16 @@ Advanced Topics memcached_verbosity memcached_version + +#################### +Deprecated Functions +#################### + +.. toctree:: + :titlesonly: + + libmemcached/memcached_fetch + ################# Platform Specific ################# @@ -98,16 +108,16 @@ Client Applications .. toctree:: :maxdepth: 1 - memcapable - memcat - memcp - memdump - memerror - memflush - memrm - memslap - memaslap - memstat + bin/memcapable + bin/memcat + bin/memcp + bin/memdump + bin/memerror + bin/memflush + bin/memrm + bin/memslap + bin/memaslap + bin/memstat ---------- Libhashkit diff --git a/docs/libmemcached/memcached_fetch.rst b/docs/libmemcached/memcached_fetch.rst new file mode 100644 index 00000000..9d995101 --- /dev/null +++ b/docs/libmemcached/memcached_fetch.rst @@ -0,0 +1,66 @@ +================= +memcached_fetch() +================= + +.. index:: object: memcached_st + +-------- +SYNOPSIS +-------- + + +#include + +.. c:function:: char *memcached_fetch(memcached_st *ptr, char *key, size_t *key_length, size_t *value_length, uint32_t *flags, memcached_return_t *error) + + .. deprecated:: 0.50 + Use :c:func:`memcached_fetch_result()` instead. + +Compile and link with -lmemcached + + +----------- +DESCRIPTION +----------- + +:c:func:`memcached_fetch()` is used to fetch an individual value from the server. :c:func:`memcached_mget()` must always be called before using this method. +You must pass in a key and its length to fetch the object. You must supply +three pointer variables which will give you the state of the returned +object. A :c:type:`uint32_t` pointer to contain whatever flags you stored with the value, a :c:type:`size_t` pointer which will be filled with size of of the +object, and a :c:type:`memcached_return_t` pointer to hold any error. The +object will be returned upon success and NULL will be returned on failure. :c:type:`MEMCACHD_END` is returned by the \*error value when all objects that have been found are returned. The final value upon :c:type:`MEMCACHED_END` is null. + +Values returned by :c:func:`memcached_fetch()` must be freed by the caller. + +All of the above functions are not tested when the +:c:type:`MEMCACHED_BEHAVIOR_USE_UDP` has been set. Executing any of these +functions with this behavior on will result in :c:type:`MEMCACHED_NOT_SUPPORTED` being returned, or for those functions which do not return a :c:type:`memcached_return_t`, the error function parameter will be set to :c:type:`MEMCACHED_NOT_SUPPORTED`. + + +------ +RETURN +------ + +:c:func:`memcached_fetch()` sets error to +to :c:type:`MEMCACHED_END` upon successful conclusion. +:c:type:`MEMCACHED_NOTFOUND` will be return if no keys at all were found. + +:c:type:`MEMCACHED_KEY_TOO_BIG` is set to error whenever :c::func:`memcached_fetch()` was used +and the key was set larger then :c:type:`MEMCACHED_MAX_KEY`, which was the largest +key allowed for the original memcached ascii server. + + +---- +HOME +---- + +To find out more information please check: +`http://libmemcached.org/ `_ + + +-------- +SEE ALSO +-------- + +:manpage:`memcached(1)` :manpage:`libmemcached(3)` :manpage:`memcached_strerror(3)` :manpage:`memcached_fetch_result(3)` + diff --git a/docs/libmemcached/memcached_return_t.rst b/docs/libmemcached/memcached_return_t.rst index 3663c6df..40f6f96f 100644 --- a/docs/libmemcached/memcached_return_t.rst +++ b/docs/libmemcached/memcached_return_t.rst @@ -25,105 +25,110 @@ SYNOPSIS :c:func:`memcached_failure()` return true if any value other then :c:type:`MEMCACHED_SUCCESS` was provided. +Libmemcached return types: +++++++++++++++++++++++++++ -.. c:type:: MEMCACHED_SUCCESS, +.. c:type:: MEMCACHED_SUCCESS Success -.. c:type:: MEMCACHED_FAILURE, +.. c:type:: MEMCACHED_FAILURE -.. c:type:: MEMCACHED_HOST_LOOKUP_FAILURE, // getaddrinfo() only +.. c:type:: MEMCACHED_HOST_LOOKUP_FAILURE -.. c:type:: MEMCACHED_CONNECTION_FAILURE, // DEPRECATED +.. c:type:: MEMCACHED_CONNECTION_FAILURE -.. c:type:: MEMCACHED_CONNECTION_BIND_FAILURE, // DEPRECATED +.. c:type:: MEMCACHED_CONNECTION_BIND_FAILURE +.. deprecated:: <0.30 -.. c:type:: MEMCACHED_WRITE_FAILURE, +.. c:type:: MEMCACHED_WRITE_FAILURE -.. c:type:: MEMCACHED_READ_FAILURE, +.. c:type:: MEMCACHED_READ_FAILURE -.. c:type:: MEMCACHED_UNKNOWN_READ_FAILURE, +.. c:type:: MEMCACHED_UNKNOWN_READ_FAILURE -.. c:type:: MEMCACHED_PROTOCOL_ERROR, +.. c:type:: MEMCACHED_PROTOCOL_ERROR -.. c:type:: MEMCACHED_CLIENT_ERROR, +.. c:type:: MEMCACHED_CLIENT_ERROR -.. c:type:: MEMCACHED_SERVER_ERROR, +.. c:type:: MEMCACHED_SERVER_ERROR -.. c:type:: MEMCACHED_CONNECTION_SOCKET_CREATE_FAILURE, // DEPRECATED +.. c:type:: MEMCACHED_CONNECTION_SOCKET_CREATE_FAILURE +.. deprecated:: <0.30 -.. c:type:: MEMCACHED_DATA_EXISTS, +.. c:type:: MEMCACHED_DATA_EXISTS -.. c:type:: MEMCACHED_DATA_DOES_NOT_EXIST, +.. c:type:: MEMCACHED_DATA_DOES_NOT_EXIST -.. c:type:: MEMCACHED_NOTSTORED, +.. c:type:: MEMCACHED_NOTSTORED -.. c:type:: MEMCACHED_STORED, +.. c:type:: MEMCACHED_STORED -.. c:type:: MEMCACHED_NOTFOUND, +.. c:type:: MEMCACHED_NOTFOUND -.. c:type:: MEMCACHED_MEMORY_ALLOCATION_FAILURE, +.. c:type:: MEMCACHED_MEMORY_ALLOCATION_FAILURE -.. c:type:: MEMCACHED_PARTIAL_READ, +.. c:type:: MEMCACHED_PARTIAL_READ -.. c:type:: MEMCACHED_SOME_ERRORS, +.. c:type:: MEMCACHED_SOME_ERRORS -.. c:type:: MEMCACHED_NO_SERVERS, +.. c:type:: MEMCACHED_NO_SERVERS -.. c:type:: MEMCACHED_END, +.. c:type:: MEMCACHED_END -.. c:type:: MEMCACHED_DELETED, +.. c:type:: MEMCACHED_DELETED -.. c:type:: MEMCACHED_VALUE, +.. c:type:: MEMCACHED_VALUE -.. c:type:: MEMCACHED_STAT, +.. c:type:: MEMCACHED_STAT -.. c:type:: MEMCACHED_ITEM, +.. c:type:: MEMCACHED_ITEM -.. c:type:: MEMCACHED_ERRNO, +.. c:type:: MEMCACHED_ERRNO -.. c:type:: MEMCACHED_FAIL_UNIX_SOCKET, // DEPRECATED +.. c:type:: MEMCACHED_FAIL_UNIX_SOCKET +.. deprecated:: <0.30 -.. c:type:: MEMCACHED_NOT_SUPPORTED, +.. c:type:: MEMCACHED_NOT_SUPPORTED -.. c:type:: MEMCACHED_NO_KEY_PROVIDED, /* Deprecated. Use MEMCACHED_BAD_KEY_PROVIDED! */ +.. c:type:: MEMCACHED_NO_KEY_PROVIDED +.. deprecated:: <0.30 + Use :c:type:`MEMCACHED_BAD_KEY_PROVIDED` instead. -.. c:type:: MEMCACHED_FETCH_NOTFINISHED, +.. c:type:: MEMCACHED_FETCH_NOTFINISHED -.. c:type:: MEMCACHED_TIMEOUT, +.. c:type:: MEMCACHED_TIMEOUT -Connection to server timed out. + Connection to server timed out. -.. c:type:: MEMCACHED_BUFFERED, +.. c:type:: MEMCACHED_BUFFERED -.. c:type:: MEMCACHED_BAD_KEY_PROVIDED, +.. c:type:: MEMCACHED_BAD_KEY_PROVIDED -.. c:type:: MEMCACHED_INVALID_HOST_PROTOCOL, +.. c:type:: MEMCACHED_INVALID_HOST_PROTOCOL -.. c:type:: MEMCACHED_SERVER_MARKED_DEAD, +.. c:type:: MEMCACHED_SERVER_MARKED_DEAD -.. c:type:: MEMCACHED_UNKNOWN_STAT_KEY, +.. c:type:: MEMCACHED_UNKNOWN_STAT_KEY -.. c:type:: MEMCACHED_E2BIG, +.. c:type:: MEMCACHED_E2BIG -.. c:type:: MEMCACHED_INVALID_ARGUMENTS, +.. c:type:: MEMCACHED_INVALID_ARGUMENTS -.. c:type:: MEMCACHED_KEY_TOO_BIG, +.. c:type:: MEMCACHED_KEY_TOO_BIG -.. c:type:: MEMCACHED_AUTH_PROBLEM, +.. c:type:: MEMCACHED_AUTH_PROBLEM -.. c:type:: MEMCACHED_AUTH_FAILURE, +.. c:type:: MEMCACHED_AUTH_FAILURE -.. c:type:: MEMCACHED_AUTH_CONTINUE, +.. c:type:: MEMCACHED_AUTH_CONTINUE -.. c:type:: MEMCACHED_PARSE_ERROR, +.. c:type:: MEMCACHED_PARSE_ERROR -.. c:type:: MEMCACHED_PARSE_USER_ERROR, +.. c:type:: MEMCACHED_PARSE_USER_ERROR -.. c:type:: MEMCACHED_DEPRECATED, - -.. c:type:: MEMCACHED_MAXIMUM_RETURN /* Always add new error code before */ +.. c:type:: MEMCACHED_DEPRECATED -------- SEE ALSO diff --git a/docs/libmemcached_configuration.rst b/docs/libmemcached_configuration.rst index 74902aab..4cfec4a8 100644 --- a/docs/libmemcached_configuration.rst +++ b/docs/libmemcached_configuration.rst @@ -2,7 +2,7 @@ Configuring Libmemcached ======================== -.. index:: object: memcached_st +.. highlightlang:: c -------- SYNOPSIS @@ -17,13 +17,12 @@ SYNOPSIS Compile and link with -lmemcached - ----------- DESCRIPTION ----------- Libmemcached implements a custom language for configuring and modifying -servers. By passing in an option string you can generate a memcached_st object +servers. By passing in an option string you can generate a ``memcached_st`` object that you can use in your application directly. .. describe:: --SERVER=:/? diff --git a/docs/memaslap.rst b/docs/memaslap.rst deleted file mode 100644 index 3f9e62dd..00000000 --- a/docs/memaslap.rst +++ /dev/null @@ -1,1248 +0,0 @@ -================================================== -memaslap - Load testing and benchmarking a server -================================================== - - --------- -SYNOPSIS --------- - -:program:: `memaslap` - -.. program:: memaslap - -.. option:: --help - ------------ -DESCRIPTION ------------ - - -memaslap is a load generation and benchmark tool for :program:`memcached()` -servers. It generates configurable workload such as threads, concurrencies, connections, run time, overwrite, miss rate, key size, value size, get/set proportion, expected throughput, and so on. Furthermore, it also testss data -verification, expire-time verification, UDP, binary protocol, facebook test, -replication test, multi-get and reconnection, etc. - -Memslap manages network connections like memcached with -libevent. Each thread of memaslap is bound with a CPU core, all -the threads don't communicate with each other, and there are several socket -connections in each thread. Each connection keeps key size distribution, -value size distribution, and command distribution by itself. - -You can specify servers via the :option:`--servers` option or via the -environment variable :envvar:`MEMCACHED_SERVERS`. - - --------- -FEATURES --------- - - -Memslap is developed to for the following purposes: - - -Manages network connections with libevent asynchronously. - - - -Set both TCP and UDP up to use non-blocking IO. - - - -Improves parallelism: higher performance in multi-threads environments. - - - -Improves time efficiency: faster processing speed. - - - -Generates key and value more efficiently; key size distribution and value size distribution are configurable. - - - -Supports get, multi-get, and set commands; command distribution is configurable. - - - -Supports controllable miss rate and overwrite rate. - - - -Supports data and expire-time verification. - - - -Supports dumping statistic information periodically. - - - -Supports thousands of TCP connections. - - - -Supports binary protocol. - - - -Supports facebook test (set with TCP and multi-get with UDP) and replication test. - - - - -------- -DETAILS -------- - - -Effective implementation of network. -____________________________________ - - -For memaslap, both TCP and UDP use non-blocking network IO. All -the network events are managed by libevent as memcached. The network module -of memaslap is similar to memcached. Libevent can ensure -memaslap can handle network very efficiently. - - -Effective implementation of multi-threads and concurrency -_________________________________________________________ - - -Memslap has the similar implementation of multi-threads to -memcached. Memslap creates one or more self-governed threads; -each thread is bound with one CPU core if the system testss setting CPU -core affinity. - -In addition, each thread has a libevent to manage the events of the network; -each thread has one or more self-governed concurrencies; and each -concurrency has one or more socket connections. All the concurrencies don’t -communicate with each other even though they are in the same thread. - -Memslap can create thousands of socket connections, and each -concurrency has tens of socket connections. Each concurrency randomly or -sequentially selects one socket connection from its socket connection pool -to run, so memaslap can ensure each concurrency handles one -socket connection at any given time. Users can specify the number of -concurrency and socket connections of each concurrency according to their -expected workload. - - -Effective implementation of generating key and value -____________________________________________________ - - -In order to improve time efficiency and space efficiency, -memaslap creates a random characters table with 10M characters. All the -suffixes of keys and values are generated from this random characters table. - -Memslap uses the offset in the character table and the length -of the string to identify a string. It can save much memory. -Each key contains two parts, a prefix and a suffix. The prefix is an -uint64_t, 8 bytes. In order to verify the data set before, -memaslap need to ensure each key is unique, so it uses the prefix to identify -a key. The prefix cannot include illegal characters, such as ‘\r’, ‘\n’, -‘\0’ and ‘ ‘. And memaslap has an algorithm to ensure that. - -Memslap doesn’t generate all the objects (key-value pairs) at -the beginning. It only generates enough objects to fill the task window -(default 10K objects) of each concurrency. Each object has the following -basic information, key prefix, key suffix offset in the character table, key -length, value offset in the character table, and value length. - -In the work process, each concurrency sequentially or randomly selects an -object from the window to do set operation or get operation. At the same -time, each concurrency kicks objects out of its window and adds new object -into it. - - -Simple but useful task scheduling -_________________________________ - - -Memslap uses libevent to schedule all the concurrencies of -threads, and each concurrency schedules tasks based on the local task -window. Memslap assumes that if each concurrency keeps the same -key distribution, value distribution and commands distribution, from -outside, memaslap keeps all the distribution as a whole. -Each task window includes a lot of objects, each object stores its basic -information, such as key, value, expire time, and so on. At any time, all -the objects in the window keep the same and fixed key and value -distribution. If an object is overwritten, the value of the object will be -updated. Memslap verifies the data or expire-time according to -the object information stored in the task window. - -Libevent selects which concurrency to handle based on a specific network -event. Then the concurrency selects which command (get or set) to operate -based on the command distribution. If it needs to kick out an old object and -add a new object, in order to keep the same key and value distribution, the -new object must have the same key length and value length. - -If memcached server has two cache layers (memory and SSD), running -memaslap with different window sizes can get different cache -miss rates. If memaslap adds enough objects into the windows at -the beginning, and the cache of memcached cannot store all the objects -initialized, then memaslap will get some objects from the second -cache layer. It causes the first cache layer to miss. So the user can -specify the window size to get the expected miss rate of the first cache -layer. - - -Useful implementation of multi-servers , UDP, TCP, multi-get and binary protocol -________________________________________________________________________________ - - -Because each thread is self-governed, memaslap can assign -different threads to handle different memcached servers. This is just one of -the ways in which memaslap testss multiple servers. The only -limitation is that the number of servers cannot be greater than the number -of threads. The other way to tests multiple servers is for replication -test. Each concurrency has one socket connection to each memcached server. -For the implementation, memaslap can set some objects to one -memcached server, and get these objects from the other servers. - -By default, Memslap does single get. If the user specifies -multi-get option, memaslap will collect enough get commands and -pack and send the commands together. - -Memslap testss both the ASCII protocol and binary protocol, -but it runs on the ASCII protocol by default. -Memslap by default runs on the TCP protocol, but it also -testss UDP. Because UDP is unreliable, dropped packages and out-of-order -packages may occur. Memslap creates a memory buffer to handle -these problems. Memslap tries to read all the response data of -one command from the server and reorders the response data. If some packages -get lost, the waiting timeout mechanism can ensure half-baked packages will -be discarded and the next command will be sent. - - - ------ -USAGE ------ - - -Below are some usage samples: - - -memaslap -s 127.0.0.1:11211 -S 5s - - - -memaslap -s 127.0.0.1:11211 -t 2m -v 0.2 -e 0.05 -b - - - -memaslap -s 127.0.0.1:11211 -F config -t 2m -w 40k -S 20s -o 0.2 - - - -memaslap -s 127.0.0.1:11211 -F config -t 2m -T 4 -c 128 -d 20 -P 40k - - - -memaslap -s 127.0.0.1:11211 -F config -t 2m -d 50 -a -n 40 - - - -memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m - - - -memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m -p 2 - - - -The user must specify one server at least to run memaslap. The -rest of the parameters have default values, as shown below: - -Thread number = 1 Concurrency = 16 - -Run time = 600 seconds Configuration file = NULL - -Key size = 64 Value size = 1024 - -Get/set = 9:1 Window size = 10k - -Execute number = 0 Single get = true - -Multi-get = false Number of sockets of each concurrency = 1 - -Reconnect = false Data verification = false - -Expire-time verification = false ASCII protocol = true - -Binary protocol = false Dumping statistic information - -periodically = false - -Overwrite proportion = 0% UDP = false - -TCP = true Limit throughput = false - -Facebook test = false Replication test = false - -Key size, value size and command distribution. -______________________________________________ - - -All the distributions are read from the configuration file specified by user -with “—cfg_cmd” option. If the user does not specify a configuration file, -memaslap will run with the default distribution (key size = 64, -value size = 1024, get/set = 9:1). For information on how to edit the -configuration file, refer to the “Configuration File” section. - -The minimum key size is 16 bytes; the maximum key size is 250 bytes. The -precision of proportion is 0.001. The proportion of distribution will be -rounded to 3 decimal places. - -The minimum value size is 1 bytes; the maximum value size is 1M bytes. The -precision of proportion is 0.001. The proportion of distribution will be -rounded to 3 decimal places. -Currently, memaslap only testss set and get commands. And it -testss 100% set and 100% get. For 100% get, it will preset some objects to -the server. - - -Multi-thread and concurrency -____________________________ - - -The high performance of memaslap benefits from the special -schedule of thread and concurrency. It’s important to specify the proper -number of them. The default number of threads is 1; the default number of -concurrency is 16. The user can use “—threads” and “--concurrency” to -specify these variables. - -If the system testss setting CPU affinity and the number of threads -specified by the user is greater than 1, memaslap will try to -bind each thread to a different CPU core. So if you want to get the best -performance memaslap, it is better to specify the number of -thread equal to the number of CPU cores. The number of threads specified by -the user can also be less or greater than the number of CPU cores. Because -of the limitation of implementation, the number of concurrencies could be -the multiple of the number of threads. - -1. For 8 CPU cores system - -For example: - ---threads=2 --concurrency=128 - ---threads=8 --concurrency=128 - ---threads=8 --concurrency=256 - ---threads=12 --concurrency=144 - -2. For 16 CPU cores system - -For example: - ---threads=8 --concurrency=128 - ---threads=16 --concurrency=256 - ---threads=16 --concurrency=512 - ---threads=24 --concurrency=288 - -The memaslap performs very well, when -used to test the performance of memcached servers. -Most of the time, the bottleneck is the network or -the server. If for some reason the user wants to -limit the performance of memaslap, there -are two ways to do this: - -Decrease the number of threads and concurrencies. -Use the option “--tps” that memaslap -provides to limit the throughput. This option allows -the user to get the expected throughput. For -example, assume that the maximum throughput is 50 -kops/s for a specific configuration, you can specify -the throughput equal to or less than the maximum -throughput using “--tps” option. - - -Window size -___________ - - -Most of the time, the user does not need to specify the window size. The -default window size is 10k. For Schooner Memcached, the user can specify -different window sizes to get different cache miss rates based on the test -case. Memslap testss cache miss rate between 0% and 100%. -If you use this utility to test the performance of Schooner Memcached, you -can specify a proper window size to get the expected cache miss rate. The -formula for calculating window size is as follows: - -Assume that the key size is 128 bytes, and the value size is 2048 bytes, and -concurrency=128. - -1. Small cache cache_size=1M, 100% cache miss (all data get from SSD). -win_size=10k - -2. cache_size=4G - -(1). cache miss rate 0% - -win_size=8k - -(2). cache miss rate 5% - -win_size=11k - -3. cache_size=16G - -(1). cache miss rate 0% - -win_size=32k - -(2). cache miss - -rate 5% - -win_size=46k - -The formula for calculating window size for cache miss rate 0%: - -cache_size / concurrency / (key_size + value_size) \* 0.5 - -The formula for calculating window size for cache miss rate 5%: - -cache_size / concurrency / (key_size + value_size) \* 0.7 - - -Verification -____________ - - -Memslap testss both data verification and expire-time -verification. The user can use "--verify=" or "-v" to specify the proportion -of data verification. In theory, it testss 100% data verification. The -user can use "--exp_verify=" or "-e" to specify the proportion of -expire-time verification. In theory, it testss 100% expire-time -verification. Specify the "--verbose" options to get more detailed error -information. - -For example: --exp_verify=0.01 –verify=0.1 , it means that 1% of the objects -set with expire-time, 10% of the objects gotten will be verified. If the -objects are gotten, memaslap will verify the expire-time and -value. - - -multi-servers and multi-config -_______________________________ - - -Memslap testss multi-servers based on self-governed thread. -There is a limitation that the number of servers cannot be greater than the -number of threads. Memslap assigns one thread to handle one -server at least. The user can use the "--servers=" or "-s" option to specify -multi-servers. - -For example: - ---servers=10.1.1.1:11211,10.1.1.2:11212,10.1.1.3:11213 --threads=6 --concurrency=36 - -The above command means that there are 6 threads, with each thread having 6 -concurrencies and that threads 0 and 3 handle server 0 (10.1.1.1); threads 1 -and 4 handle server 1 (10.1.1.2); and thread 2 and 5 handle server 2 -(10.1.1.3). - -All the threads and concurrencies in memaslap are self-governed. - -So is memaslap. The user can start up several -memaslap instances. The user can run memaslap on different client -machines to communicate with the same memcached server at the same. It is -recommended that the user start different memaslap on different -machines using the same configuration. - - -Run with execute number mode or time mode -_________________________________________ - - -The default memaslap runs with time mode. The default run time -is 10 minutes. If it times out, memaslap will exit. Do not -specify both execute number mode and time mode at the same time; just -specify one instead. - -For example: - ---time=30s (It means the test will run 30 seconds.) - ---execute_number=100000 (It means that after running 100000 commands, the test will exit.) - - -Dump statistic information periodically. -________________________________________ - - -The user can use "--stat_freq=" or "-S" to specify the frequency. - -For example: - ---stat_freq=20s - -Memslap will dump the statistics of the commands (get and set) at the frequency of every 20 -seconds. - -For more information on the format of dumping statistic information, refer to “Format of Output” section. - - -Multi-get -_________ - - -The user can use "--division=" or "-d" to specify multi-get keys count. -Memslap by default does single get with TCP. Memslap also testss data -verification and expire-time verification for multi-get. - -Memslap testss multi-get with both TCP and UDP. Because of -the different implementation of the ASCII protocol and binary protocol, -there are some differences between the two. For the ASCII protocol, -memaslap sends one “multi-get” to the server once. For the -binary protocol, memaslap sends several single get commands -together as “multi-get” to the server. - - -UDP and TCP -___________ - - -Memslap testss both UDP and TCP. For TCP, -memaslap does not reconnect the memcached server if socket connections are -lost. If all the socket connections are lost or memcached server crashes, -memaslap will exit. If the user specifies the “--reconnect” -option when socket connections are lost, it will reconnect them. - -User can use “--udp” to enable the UDP feature, but UDP comes with some -limitations: - -UDP cannot set data more than 1400 bytes. - -UDP is not testsed by the binary protocol because the binary protocol of -memcached does not tests that. - -UDP doesn’t tests reconnection. - - -Facebook test -_____________ - - -Set data with TCP and multi-get with UDP. Specify the following options: - -"--facebook --division=50" - -If you want to create thousands of TCP connections, specify the - -"--conn_sock=" option. - -For example: --facebook --division=50 --conn_sock=200 - -The above command means that memaslap will do facebook test, -each concurrency has 200 socket TCP connections and one UDP socket. - -Memslap sets objects with the TCP socket, and multi-gets 50 -objects once with the UDP socket. - -If you specify "--division=50", the key size must be less that 25 bytes -because the UDP packet size is 1400 bytes. - - -Replication test -________________ - - -For replication test, the user must specify at least two memcached servers. -The user can use “—rep_write=” option to enable feature. - -For example: - ---servers=10.1.1.1:11211,10.1.1.2:11212 –rep_write=2 - -The above command means that there are 2 replication memcached servers, -memaslap will set objects to both server 0 and server 1, get -objects which are set to server 0 before from server 1, and also get objects -which are set to server 1 before from server 0. If server 0 crashes, -memaslap will only get objects from server 1. If server 0 comes -back to life again, memaslap will reconnect server 0. If both -server 0 and server 1 crash, memaslap will exit. - - -Supports thousands of TCP connections -_____________________________________ - - -Start memaslap with "--conn_sock=" or "-n" to enable this -feature. Make sure that your system can tests opening thousands of files -and creating thousands of sockets. However, this feature does not tests -reconnection if sockets disconnect. - -For example: - ---threads=8 --concurrency=128 --conn_sock=128 - -The above command means that memaslap starts up 8 threads, each -thread has 16 concurrencies, each concurrency has 128 TCP socket -connections, and the total number of TCP socket connections is 128 \* 128 = -16384. - - -Supports binary protocol -________________________ - - -Start memaslap with "--binary" or "-B" options to enable this -feature. It testss all the above features except UDP, because the latest -memcached 1.3.3 does not implement binary UDP protocol. - -For example: - ---binary - -Since memcached 1.3.3 doesn't implement binary UDP protocol, -memaslap does not tests UDP. In addition, memcached 1.3.3 does not tests -multi-get. If you specify "--division=50" option, it just sends 50 get -commands together as “mulit-get” to the server. - - - ------------------- -Configuration file ------------------- - - -This section describes the format of the configuration file. By default -when no configuration file is specified memaslap reads the default -one located at ~/.memaslap.cnf. - -Below is a sample configuration file: - - -.. code-block:: perl - - --------------------------------------------------------------------------- - #comments should start with '#' - #key - #start_len end_len proportion - # - #key length range from start_len to end_len - #start_len must be equal to or greater than 16 - #end_len must be equal to or less than 250 - #start_len must be equal to or greater than end_len - #memaslap will generate keys according to the key range - #proportion: indicates keys generated from one range accounts for the total - generated keys - # - #example1: key range 16~100 accounts for 80% - # key range 101~200 accounts for 10% - # key range 201~250 accounts for 10% - # total should be 1 (0.8+0.1+0.1 = 1) - # - # 16 100 0.8 - # 101 200 0.1 - # 201 249 0.1 - # - #example2: all keys length are 128 bytes - # - # 128 128 1 - key - 128 128 1 - #value - #start_len end_len proportion - # - #value length range from start_len to end_len - #start_len must be equal to or greater than 1 - #end_len must be equal to or less than 1M - #start_len must be equal to or greater than end_len - #memaslap will generate values according to the value range - #proportion: indicates values generated from one range accounts for the - total generated values - # - #example1: value range 1~1000 accounts for 80% - # value range 1001~10000 accounts for 10% - # value range 10001~100000 accounts for 10% - # total should be 1 (0.8+0.1+0.1 = 1) - # - # 1 1000 0.8 - # 1001 10000 0.1 - # 10001 100000 0.1 - # - #example2: all value length are 128 bytes - # - # 128 128 1 - value - 2048 2048 1 - #cmd - #cmd_type cmd_proportion - # - #currently memaslap only testss get and set command. - # - #cmd_type - #set 0 - #get 1 - # - #example: set command accounts for 50% - # get command accounts for 50% - # total should be 1 (0.5+0.5 = 1) - # - # cmd - # 0 0.5 - # 1 0.5 - cmd - 0 0.1 - 1.0 0.9 - - - ----------------- -Format of output ----------------- - - -At the beginning, memaslap displays some configuration information as follows: - - -servers : 127.0.0.1:11211 - - - -threads count: 1 - - - -concurrency: 16 - - - -run time: 20s - - - -windows size: 10k - - - -set proportion: set_prop=0.10 - - - -get proportion: get_prop=0.90 - - - -Where -_____ - - - -servers : "servers" - - The servers used by memaslap. - - - -threads count - - The number of threads memaslap runs with. - - - -concurrency - - The number of concurrencies memaslap runs with. - - - -run time - - How long to run memaslap. - - - -windows size - - The task window size of each concurrency. - - - -set proportion - - The proportion of set command. - - - -get proportion - - The proportion of get command. - - - -The output of dynamic statistics is something like this: - - -.. code-block:: perl - - --------------------------------------------------------------------------------------------------------------------------------- - Get Statistics - Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) - Avg(us) Std_dev Geo_dist - Period 5 345826 69165 65.3 0 27 2198 203 - 95.43 177.29 - Global 20 1257935 62896 71.8 0 26 3791 224 - 117.79 192.60 - - - Set Statistics - Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) - Avg(us) Std_dev Geo_dist - Period 5 38425 7685 7.3 0 42 628 240 - 88.05 220.21 - Global 20 139780 6989 8.0 0 37 3790 253 - 117.93 224.83 - - - Total Statistics - Type Time(s) Ops TPS(ops/s) Net(M/s) Get_miss Min(us) Max(us) - Avg(us) Std_dev Geo_dist - Period 5 384252 76850 72.5 0 27 2198 207 - 94.72 181.18 - Global 20 1397720 69886 79.7 0 26 3791 227 - 117.93 195.60 - --------------------------------------------------------------------------------------------------------------------------------- - - - -Where -_____ - - - -Get Statistics - - Statistics information of get command - - - -Set Statistics - - Statistics information of set command - - - -Total Statistics - - Statistics information of both get and set command - - - -Period - - Result within a period - - - -Global - - Accumulated results - - - -Ops - - Total operations - - - -TPS - - Throughput, operations/second - - - -Net - - The rate of network - - - -Get_miss - - How many objects can’t be gotten - - - -Min - - The minimum response time - - - -Max - - The maximum response time - - - -Avg: - - The average response time - - - -Std_dev - - Standard deviation of response time - - - -Geo_dist - - Geometric distribution based on natural exponential function - - - -At the end, memaslap will output something like this: - - -.. code-block:: perl - - --------------------------------------------------------------------------------------------------------------------------------- - Get Statistics (1257956 events) - Min: 26 - Max: 3791 - Avg: 224 - Geo: 192.60 - Std: 116.23 - Log2 Dist: - 4: 0 10 84490 215345 - 8: 484890 459823 12543 824 - 12: 31 - - Set Statistics (139782 events) - Min: 37 - Max: 3790 - Avg: 253 - Geo: 224.84 - Std: 116.83 - Log2 Dist: - 4: 0 0 4200 16988 - 8: 50784 65574 2064 167 - 12: 5 - - Total Statistics (1397738 events) - Min: 26 - Max: 3791 - Avg: 227 - Geo: 195.60 - Std: 116.60 - Log2 Dist: - 4: 0 10 88690 232333 - 8: 535674 525397 14607 991 - 12: 36 - - cmd_get: 1257969 - cmd_set: 139785 - get_misses: 0 - verify_misses: 0 - verify_failed: 0 - expired_get: 0 - unexpired_unget: 0 - written_bytes: 242516030 - read_bytes: 1003702556 - object_bytes: 152086080 - packet_disorder: 0 - packet_drop: 0 - udp_timeout: 0 - - Run time: 20.0s Ops: 1397754 TPS: 69817 Net_rate: 59.4M/s - --------------------------------------------------------------------------------------------------------------------------------- - - - -Where -_____ - - - -Get Statistics - - Get statistics of response time - - - -Set Statistics - - Set statistics of response time - - - -Total Statistics - - Both get and set statistics of response time - - - -Min - - The accumulated and minimum response time - - - -Max - - The accumulated and maximum response time - - - -Avg - - The accumulated and average response time - - - -Std - - Standard deviation of response time - - - -Log2 Dist - - Geometric distribution based on logarithm 2 - - - -cmd_get - - Total get commands done - - - -cmd_set - - Total set commands done - - - -get_misses - - How many objects can’t be gotten from server - - - -verify_misses - - How many objects need to verify but can’t get them - - - -verify_failed - - How many objects with insistent value - - - -expired_get - - How many objects are expired but we get them - - - -unexpired_unget - - How many objects are unexpired but we can’t get them - - - -written_bytes - - Total written bytes - - - -read_bytes - - Total read bytes - - - -object_bytes - - Total object bytes - - - -packet_disorder - - How many UDP packages are disorder - - - -packet_drop - - How many UDP packages are lost - - - -udp_timeout - - How many times UDP time out happen - - - -Run time - - Total run time - - - -Ops - - Total operations - - - -TPS - - Throughput, operations/second - - - -Net_rate - - The average rate of network - - - - - -------- -OPTIONS -------- - - --s, --servers= - List one or more servers to connect. Servers count must be less than - threads count. e.g.: --servers=localhost:1234,localhost:11211 - --T, --threads= - Number of threads to startup, better equal to CPU numbers. Default 8. - --c, --concurrency= - Number of concurrency to simulate with load. Default 128. - --n, --conn_sock= - Number of TCP socks per concurrency. Default 1. - --x, --execute_number= - Number of operations(get and set) to execute for the - given test. Default 1000000. - --t, --time= - How long the test to run, suffix: s-seconds, m-minutes, h-hours, - d-days e.g.: --time=2h. - --F, --cfg_cmd= - Load the configure file to get command,key and value distribution list. - --w, --win_size= - Task window size of each concurrency, suffix: K, M e.g.: --win_size=10k. - Default 10k. - --X, --fixed_size= - Fixed length of value. - --v, --verify= - The proportion of date verification, e.g.: --verify=0.01 - --d, --division= - Number of keys to multi-get once. Default 1, means single get. - --S, --stat_freq= - Frequency of dumping statistic information. suffix: s-seconds, - m-minutes, e.g.: --resp_freq=10s. - --e, --exp_verify= - The proportion of objects with expire time, e.g.: --exp_verify=0.01. - Default no object with expire time - --o, --overwrite= - The proportion of objects need overwrite, e.g.: --overwrite=0.01. - Default never overwrite object. - --R, --reconnect - Reconnect tests, when connection is closed it will be reconnected. - --U, --udp - UDP tests, default memaslap uses TCP, TCP port and UDP port of - server must be same. - --a, --facebook - Whether it enables facebook test feature, set with TCP and multi-get with UDP. - --B, --binary - Whether it enables binary protocol. Default with ASCII protocol. - --P, --tps= - Expected throughput, suffix: K, e.g.: --tps=10k. - --p, --rep_write= - The first nth servers can write data, e.g.: --rep_write=2. - --b, --verbose - Whether it outputs detailed information when verification fails. - --h, --help - Display this message and then exit. - --V, --version - Display the version of the application and then exit. - - --------- -EXAMPLES --------- - - -memaslap -s 127.0.0.1:11211 -S 5s - -memaslap -s 127.0.0.1:11211 -t 2m -v 0.2 -e 0.05 -b - -memaslap -s 127.0.0.1:11211 -F config -t 2m -w 40k -S 20s -o 0.2 - -memaslap -s 127.0.0.1:11211 -F config -t 2m -T 4 -c 128 -d 20 -P 40k - -memaslap -s 127.0.0.1:11211 -F config -t 2m -d 50 -a -n 40 - -memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m - -memaslap -s 127.0.0.1:11211,127.0.0.1:11212 -F config -t 2m -p 2 - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - -------- -AUTHORS -------- - - -Mingqiang Zhuang (Schooner Technolgy) -Brian Aker, - - --------- -SEE ALSO --------- - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/memcached_behavior.rst b/docs/memcached_behavior.rst index f7acb297..59ec4839 100644 --- a/docs/memcached_behavior.rst +++ b/docs/memcached_behavior.rst @@ -39,7 +39,7 @@ on or off options you just need to pass in a value of 1. Calls to :c:func:`memcached_behavior_set()` will flush and reset all connections. -.. c:var:: MEMCACHED_BEHAVIOR_USE_UDP +.. c:type:: MEMCACHED_BEHAVIOR_USE_UDP Causes :manpage:`libmemcached(3)` to use the UDP transport when communicating with a memcached server. Not all I/O operations are testsed @@ -58,18 +58,18 @@ not been enabled. -.. c:var:: MEMCACHED_BEHAVIOR_NO_BLOCK +.. c:type:: MEMCACHED_BEHAVIOR_NO_BLOCK Causes :manpage:`libmemcached(3)` to use asychronous IO. This is the fastest transport available for storage functions. -.. c:var:: MEMCACHED_BEHAVIOR_SND_TIMEOUT +.. c:type:: MEMCACHED_BEHAVIOR_SND_TIMEOUT This sets the microsecond behavior of the socket against the SO_SNDTIMEO flag. In cases where you cannot use non-blocking IO this will allow you to still have timeouts on the sending of data. -.. c:var:: MEMCACHED_BEHAVIOR_RCV_TIMEOUT +.. c:type:: MEMCACHED_BEHAVIOR_RCV_TIMEOUT This sets the microsecond behavior of the socket against the SO_RCVTIMEO flag. @@ -77,14 +77,14 @@ In cases where you cannot use non-blocking IO this will allow you to still have -.. c:var:: MEMCACHED_BEHAVIOR_TCP_NODELAY +.. c:type:: MEMCACHED_BEHAVIOR_TCP_NODELAY Turns on the no-delay feature for connecting sockets (may be faster in some environments). -.. c:var:: MEMCACHED_BEHAVIOR_HASH +.. c:type:: MEMCACHED_BEHAVIOR_HASH Makes the default hashing algorithm for keys use MD5. The value can be set to either :c:type:`MEMCACHED_HASH_DEFAULT`, :c:type:`MEMCACHED_HASH_MD5`, :c:type:`MEMCACHED_HASH_CRC`, :c:type:`MEMCACHED_HASH_FNV1_64`, :c:type:`MEMCACHED_HASH_FNV1A_64`, :c:type:`MEMCACHED_HASH_FNV1_32`, :c:type:`MEMCACHED_HASH_FNV1A_32`, :c:type:`MEMCACHED_HASH_JENKINS`, :c:type:`MEMCACHED_HASH_HSIEH`, and :c:type:`MEMCACHED_HASH_MURMUR`. @@ -95,162 +95,167 @@ Support for :c:type:`MEMCACHED_HASH_HSIEH` is a compile time option that is disa -.. c:var:: MEMCACHED_BEHAVIOR_DISTRIBUTION +.. c:type:: MEMCACHED_BEHAVIOR_DISTRIBUTION Using this you can enable different means of distributing values to servers. The default method is MEMCACHED_DISTRIBUTION_MODULA. You can enable consistent hashing by setting MEMCACHED_DISTRIBUTION_CONSISTENT. Consistent hashing delivers better distribution and allows servers to be added to the cluster with minimal cache losses. Currently MEMCACHED_DISTRIBUTION_CONSISTENT is an alias for the value MEMCACHED_DISTRIBUTION_CONSISTENT_KETAMA. -.. c:var:: MEMCACHED_BEHAVIOR_CACHE_LOOKUPS +.. c:type:: MEMCACHED_BEHAVIOR_CACHE_LOOKUPS +.. deprecated:: 0.46(?) + DNS lookups are now always cached until an error occurs with the server. -DEPRECATED. Memcached can cache named lookups so that DNS lookups are made only once. + Memcached can cache named lookups so that DNS lookups are made only once. -.. c:var:: MEMCACHED_BEHAVIOR_SUPPORT_CAS +.. c:type:: MEMCACHED_BEHAVIOR_SUPPORT_CAS Support CAS operations (this is not enabled by default at this point in the server since it imposes a slight performance penalty). -.. c:var:: MEMCACHED_BEHAVIOR_KETAMA +.. c:type:: MEMCACHED_BEHAVIOR_KETAMA Sets the default distribution to MEMCACHED_DISTRIBUTION_CONSISTENT_KETAMA and the hash to MEMCACHED_HASH_MD5. -.. c:var:: MEMCACHED_BEHAVIOR_KETAMA_WEIGHTED +.. c:type:: MEMCACHED_BEHAVIOR_KETAMA_WEIGHTED + + Sets the default distribution to MEMCACHED_DISTRIBUTION_CONSISTENT_KETAMA with the weighted tests. and the hash to MEMCACHED_HASH_MD5. -Sets the default distribution to MEMCACHED_DISTRIBUTION_CONSISTENT_KETAMA with the weighted tests. and the hash to MEMCACHED_HASH_MD5. - -.. c:var:: MEMCACHED_BEHAVIOR_KETAMA_HASH +.. c:type:: MEMCACHED_BEHAVIOR_KETAMA_HASH Sets the hashing algorithm for host mapping on continuum. The value can be set to either MEMCACHED_HASH_DEFAULT, MEMCACHED_HASH_MD5, MEMCACHED_HASH_CRC, MEMCACHED_HASH_FNV1_64, MEMCACHED_HASH_FNV1A_64, MEMCACHED_HASH_FNV1_32, and MEMCACHED_HASH_FNV1A_32. -.. c:var:: MEMCACHED_BEHAVIOR_KETAMA_COMPAT +.. c:type:: MEMCACHED_BEHAVIOR_KETAMA_COMPAT Sets the compatibility mode. The value can be set to either MEMCACHED_KETAMA_COMPAT_LIBMEMCACHED (this is the default) or MEMCACHED_KETAMA_COMPAT_SPY to be compatible with the SPY Memcached client for Java. -.. c:var:: MEMCACHED_BEHAVIOR_POLL_TIMEOUT +.. c:type:: MEMCACHED_BEHAVIOR_POLL_TIMEOUT Modify the timeout value that is used by poll(). The default value is -1. An signed int pointer must be passed to memcached_behavior_set() to change this value. For memcached_behavior_get() a signed int value will be cast and returned as the unsigned long long. -.. c:var:: MEMCACHED_BEHAVIOR_USER_DATA - -DEPRECATED +.. c:type:: MEMCACHED_BEHAVIOR_USER_DATA +.. deprecated:: < 0.30 -.. c:var:: MEMCACHED_BEHAVIOR_BUFFER_REQUESTS +.. c:type:: MEMCACHED_BEHAVIOR_BUFFER_REQUESTS Enabling buffered IO causes commands to "buffer" instead of being sent. Any action that gets data causes this buffer to be be sent to the remote connection. Quiting the connection or closing down the connection will also cause the buffered data to be pushed to the remote connection. -.. c:var:: MEMCACHED_BEHAVIOR_VERIFY_KEY +.. c:type:: MEMCACHED_BEHAVIOR_VERIFY_KEY Enabling this will cause :manpage:`libmemcached(3)` to test all keys to verify that they are valid keys. -.. c:var:: MEMCACHED_BEHAVIOR_SORT_HOSTS +.. c:type:: MEMCACHED_BEHAVIOR_SORT_HOSTS Enabling this will cause hosts that are added to be placed in the host list in sorted order. This will defeat consisten hashing. -.. c:var:: MEMCACHED_BEHAVIOR_CONNECT_TIMEOUT +.. c:type:: MEMCACHED_BEHAVIOR_CONNECT_TIMEOUT In non-blocking mode this changes the value of the timeout during socket connection. -.. c:var:: MEMCACHED_BEHAVIOR_BINARY_PROTOCOL +.. c:type:: MEMCACHED_BEHAVIOR_BINARY_PROTOCOL Enable the use of the binary protocol. Please note that you cannot toggle this flag on an open connection. -.. c:var:: MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT +.. c:type:: MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT Set this value to enable the server be removed after continuous MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT times connection failure. -.. c:var:: MEMCACHED_BEHAVIOR_IO_MSG_WATERMARK +.. c:type:: MEMCACHED_BEHAVIOR_IO_MSG_WATERMARK Set this value to tune the number of messages that may be sent before libmemcached should start to automatically drain the input queue. Setting this value to high, may cause libmemcached to deadlock (trying to send data, but the send will block because the input buffer in the kernel is full). -.. c:var:: MEMCACHED_BEHAVIOR_IO_BYTES_WATERMARK +.. c:type:: MEMCACHED_BEHAVIOR_IO_BYTES_WATERMARK Set this value to tune the number of bytes that may be sent before libmemcached should start to automatically drain the input queue (need at least 10 IO requests sent without reading the input buffer). Setting this value to high, may cause libmemcached to deadlock (trying to send data, but the send will block because the input buffer in the kernel is full). -.. c:var:: MEMCACHED_BEHAVIOR_IO_KEY_PREFETCH +.. c:type:: MEMCACHED_BEHAVIOR_IO_KEY_PREFETCH The binary protocol works a bit different than the textual protocol in that a multiget is implemented as a pipe of single get-operations which are sent to the server in a chunk. If you are using large multigets from your application, you may improve the latency of the gets by setting this value so you send out the first chunk of requests when you hit the specified limit. It allows the servers to start processing the requests to send the data back while the rest of the requests are created and sent to the server. -.. c:var:: MEMCACHED_BEHAVIOR_NOREPLY +.. c:type:: MEMCACHED_BEHAVIOR_NOREPLY Set this value to specify that you really don't care about the result from your storage commands (set, add, replace, append, prepend). -.. c:var:: MEMCACHED_BEHAVIOR_NUMBER_OF_REPLICAS +.. c:type:: MEMCACHED_BEHAVIOR_NUMBER_OF_REPLICAS If you just want "a poor mans HA", you may specify the numbers of replicas libmemcached should store of each item (on different servers). This replication does not dedicate certain memcached servers to store the replicas in, but instead it will store the replicas together with all of the other objects (on the 'n' next servers specified in your server list). -.. c:var:: MEMCACHED_BEHAVIOR_RANDOMIZE_REPLICA_READ +.. c:type:: MEMCACHED_BEHAVIOR_RANDOMIZE_REPLICA_READ Allows randomizing the replica reads starting point. Normally the read is done from primary server and in case of miss the read is done from primary + 1, then primary + 2 all the way to 'n' replicas. If this option is set on the starting point of the replica reads is randomized between the servers. This allows distributing read load to multiple servers with the expense of more write traffic. -.. c:var:: MEMCACHED_BEHAVIOR_CORK +.. c:type:: MEMCACHED_BEHAVIOR_CORK This open has been deprecated with the behavior now built and used appropriately on selected platforms. -.. c:var:: MEMCACHED_BEHAVIOR_KEEPALIVE +.. c:type:: MEMCACHED_BEHAVIOR_KEEPALIVE Enable TCP_KEEPALIVE behavior. -.. c:var:: MEMCACHED_BEHAVIOR_KEEPALIVE_IDLE +.. c:type:: MEMCACHED_BEHAVIOR_KEEPALIVE_IDLE Specify time, in seconds, to mark a connection as idle. This is only available as an option Linux. -.. c:var:: MEMCACHED_BEHAVIOR_SOCKET_SEND_SIZE +.. c:type:: MEMCACHED_BEHAVIOR_SOCKET_SEND_SIZE Find the current size of SO_SNDBUF. A value of 0 means either an error occured or no hosts were available. It is safe to assume system default if this occurs. If an error occurs you can checked the last cached errno statement to find the specific error. -.. c:var:: MEMCACHED_BEHAVIOR_SOCKET_RECV_SIZE +.. c:type:: MEMCACHED_BEHAVIOR_SOCKET_RECV_SIZE Find the current size of SO_RCVBUF. A value of 0 means either an error occured or no hosts were available. It is safe to assume system default if this occurs. If an error occurs you can checked the last cached errno statement to find the specific error. -.. c:var:: MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT +.. c:type:: MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT +.. deprecated:: 0.48 + See :c:type:`MEMCACHED_BEHAVIOR_REMOVE_FAILED_SERVERS` -DEPRECATED, please see MEMCACHED_BEHAVIOR_REMOVE_FAILED_SERVERS. This number of times a host can have an error before it is disabled. + This number of times a host can have an error before it is disabled. -.. c:var:: MEMCACHED_BEHAVIOR_AUTO_EJECT_HOSTS +.. c:type:: MEMCACHED_BEHAVIOR_AUTO_EJECT_HOSTS +.. deprecated:: 0.48 + See :c:type:`MEMCACHED_BEHAVIOR_REMOVE_FAILED_SERVERS` -DEPRECATED, please see MEMCACHED_BEHAVIOR_REMOVE_FAILED_SERVERS. If enabled any hosts which have been flagged as disabled will be removed from the list of servers in the memcached_st structure. This must be used in combination with MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT. + If enabled any hosts which have been flagged as disabled will be removed from the list of servers in the memcached_st structure. This must be used in combination with MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT. -.. c:var:: MEMCACHED_BEHAVIOR_REMOVE_FAILED_SERVERS +.. c:type:: MEMCACHED_BEHAVIOR_REMOVE_FAILED_SERVERS -If enabled any hosts which have been flagged as disabled will be removed from the list of servers in the memcached_st structure. + If enabled any hosts which have been flagged as disabled will be removed from the list of servers in the memcached_st structure. -.. c:var:: MEMCACHED_BEHAVIOR_RETRY_TIMEOUT +.. c:type:: MEMCACHED_BEHAVIOR_RETRY_TIMEOUT When enabled a host which is problematic will only be checked for usage based on the amount of time set by this behavior. -.. c:var:: MEMCACHED_BEHAVIOR_HASH_WITH_PREFIX_KEY +.. c:type:: MEMCACHED_BEHAVIOR_HASH_WITH_PREFIX_KEY When enabled the prefix key will be added to the key when determining server by hash. diff --git a/docs/memcached_callback.rst b/docs/memcached_callback.rst index 3cdd5adb..e5723d90 100644 --- a/docs/memcached_callback.rst +++ b/docs/memcached_callback.rst @@ -40,21 +40,19 @@ You can use :c:type:`MEMCACHED_CALLBACK_USER_DATA` to provide custom context if required for any of the callbacks. -.. c:var:: MEMCACHED_CALLBACK_CLEANUP_FUNCTION +.. c:type:: MEMCACHED_CALLBACK_CLEANUP_FUNCTION When :c:func:`memcached_delete()` is called this function will be excuted. At the point of its execution all connections are closed. -.. c:var:: MEMCACHED_CALLBACK_CLONE_FUNCTION +.. c:type:: MEMCACHED_CALLBACK_CLONE_FUNCTION When :c:func:`memcached_delete()` is called this function will be excuted. At the point of its execution all connections are closed. - - -.. c:var:: MEMCACHED_CALLBACK_PREFIX_KEY +.. c:type:: MEMCACHED_CALLBACK_PREFIX_KEY You can set a value which will be used to create a domain for your keys. The value specified here will be prefixed to each of your keys. The value can @@ -67,34 +65,25 @@ a key which is too long, :c:type:`MEMCACHED_BAD_KEY_PROVIDED` will be returned. If you set a value with the value being NULL then the prefix key is disabled. - -.. c:var:: MEMCACHED_CALLBACK_USER_DATA +.. c:type:: MEMCACHED_CALLBACK_USER_DATA This allows you to store a pointer to a specifc piece of data. This can be retrieved from inside of :c:func:`memcached_fetch_execute()`. Cloning a :c:type:`memcached_st` will copy the pointer to the clone. +.. c:type:: MEMCACHED_CALLBACK_MALLOC_FUNCTION +.. deprecated:: <0.32 + Use :c:type:`memcached_set_memory_allocators` instead. - -.. c:var:: MEMCACHED_CALLBACK_MALLOC_FUNCTION - -DEPRECATED: use :c:type:`memcached_set_memory_allocators` instead. - - - -.. c:var:: MEMCACHED_CALLBACK_REALLOC_FUNCTION - -DEPRECATED: use :c:type:`memcached_set_memory_allocators` instead. +.. c:type:: MEMCACHED_CALLBACK_REALLOC_FUNCTION +.. deprecated:: <0.32 + Use :c:type:`memcached_set_memory_allocators` instead. +.. c:type:: MEMCACHED_CALLBACK_FREE_FUNCTION +.. deprecated:: <0.32 + Use :c:type:`memcached_set_memory_allocators` instead. - -.. c:var:: MEMCACHED_CALLBACK_FREE_FUNCTION - -DEPRECATED: use :c:type:`memcached_set_memory_allocators` instead. - - - -.. c:var:: MEMCACHED_CALLBACK_GET_FAILURE +.. c:type:: MEMCACHED_CALLBACK_GET_FAILURE This function implements the read through cache behavior. On failure of retrieval this callback will be called. @@ -113,7 +102,7 @@ The prototype for this is: -.. c:var:: MEMCACHED_CALLBACK_DELETE_TRIGGER +.. c:type:: MEMCACHED_CALLBACK_DELETE_TRIGGER This function implements a trigger upon successful deletion of a key. The memcached_st structure will need to be cloned in order to make use of it. diff --git a/docs/memcached_create.rst b/docs/memcached_create.rst index 782b1a06..8e1d30f5 100644 --- a/docs/memcached_create.rst +++ b/docs/memcached_create.rst @@ -2,21 +2,21 @@ Creating and destroying a memcached_st ====================================== -.. index:: object: memcached_st - -------- SYNOPSIS -------- #include + +.. c:type:: memcachd_st + +.. c:function:: memcached_st* memcached_create(memcached_st *ptr) -.. c:function:: memcached_st *memcached_create (memcached_st *ptr); - -.. c:function:: void memcached_free (memcached_st *ptr); +.. c:function:: void memcached_free(memcached_st *ptr) -.. c:function:: memcached_st *memcached_clone (memcached_st *destination, memcached_st *source); +.. c:function:: memcached_st* memcached_clone(memcached_st *destination, memcached_st *source) -.. c:function:: void memcached_servers_reset(memcached_st); +.. c:function:: void memcached_servers_reset(memcached_st) Compile and link with -lmemcached diff --git a/docs/memcached_get.rst b/docs/memcached_get.rst index d3f5806f..4285b3ae 100644 --- a/docs/memcached_get.rst +++ b/docs/memcached_get.rst @@ -11,23 +11,21 @@ SYNOPSIS #include -.. c:function:: memcached_result_st * memcached_fetch_result (memcached_st *ptr, memcached_result_st *result, memcached_return_t *error); +.. c:function:: memcached_result_st * memcached_fetch_result (memcached_st *ptr, memcached_result_st *result, memcached_return_t *error) -.. c:function:: char * memcached_get (memcached_st *ptr, const char *key, size_t key_length, size_t *value_length, uint32_t *flags, memcached_return_t *error); +.. c:function:: char * memcached_get (memcached_st *ptr, const char *key, size_t key_length, size_t *value_length, uint32_t *flags, memcached_return_t *error) -.. c:function:: memcached_return_t memcached_mget (memcached_st *ptr, const char * const *keys, const size_t *key_length, size_t number_of_keys); +.. c:function:: memcached_return_t memcached_mget (memcached_st *ptr, const char * const *keys, const size_t *key_length, size_t number_of_keys) -.. c:function:: char * memcached_get_by_key (memcached_st *ptr, const char *group_key, size_t group_key_length, const char *key, size_t key_length, size_t *value_length, uint32_t *flags, memcached_return_t *error); +.. c:function:: char * memcached_get_by_key (memcached_st *ptr, const char *group_key, size_t group_key_length, const char *key, size_t key_length, size_t *value_length, uint32_t *flags, memcached_return_t *error) -.. c:function:: memcached_return_t memcached_mget_by_key (memcached_st *ptr, const char *group_key, size_t group_key_length, const char * const *keys, const size_t *key_length, size_t number_of_keys); +.. c:function:: memcached_return_t memcached_mget_by_key (memcached_st *ptr, const char *group_key, size_t group_key_length, const char * const *keys, const size_t *key_length, size_t number_of_keys) -.. c:function:: char * memcached_fetch (memcached_st *ptr, char *key, size_t *key_length, size_t *value_length, uint32_t *flags, memcached_return_t *error); +.. c:function:: memcached_return_t memcached_fetch_execute (memcached_st *ptr, memcached_execute_fn *callback, void *context, uint32_t number_of_callbacks) -.. c:function:: memcached_return_t memcached_fetch_execute (memcached_st *ptr, memcached_execute_fn *callback, void *context, uint32_t number_of_callbacks); +.. c:function:: memcached_return_t memcached_mget_execute (memcached_st *ptr, const char * const *keys, const size_t *key_length, size_t number_of_keys, memcached_execute_fn *callback, void *context, uint32_t number_of_callbacks) -.. c:function:: memcached_return_t memcached_mget_execute (memcached_st *ptr, const char * const *keys, const size_t *key_length, size_t number_of_keys, memcached_execute_fn *callback, void *context, uint32_t number_of_callbacks); - -.. c:function:: memcached_return_t memcached_mget_execute_by_key (memcached_st *ptr, const char *group_key, size_t group_key_length, const char * const *keys, const size_t *key_length, size_t number_of_keys, memcached_execute_fn *callback, void *context, uint32_t number_of_callbacks); +.. c:function:: memcached_return_t memcached_mget_execute_by_key (memcached_st *ptr, const char *group_key, size_t group_key_length, const char * const *keys, const size_t *key_length, size_t number_of_keys, memcached_execute_fn *callback, void *context, uint32_t number_of_callbacks) Compile and link with -lmemcached @@ -47,23 +45,19 @@ object returned by :c:func:`memcached_get()` must be released by the caller application. :c:func:`memcached_mget()` is used to select multiple keys at once. For -multiple key operations it is always faster to use this function. This function always works asynchronously. :c:func:`memcached_fetch()` is then used to -retrieve any keys found. No error is given on keys that are not found. You -must call either :c:func:`memcached_fetch()` or -:c:func:`memcached_fetch_result()` after a successful call to -:c:func:`memcached_mget()`. You should continue to call these functions until -they return NULL (aka no more values). If you need to quit in the middle of a -:c:func:`memcached_get()` call, execute a :c:func:`memcached_quit()`. After you do this, you can issue new queries against the server. - -:c:func:`memcached_fetch()` is used to fetch an individual value from the server. :c:func:`memcached_mget()` must always be called before using this method. +multiple key operations it is always faster to use this function. This function always works asynchronously. + +To retrieve data after a successful execution of :c:func:`memcached_mget()`, you will need to +call :c:func:`memcached_fetch_result()`. You should continue to call this function until +it returns a NULL (i.e. no more values). If you need to quit in the middle of a +:c:func:`memcached_mget()` call, you can execute a :c:func:`memcached_quit()`, those this is not required. + +:c:func:`memcached_fetch_result()` is used to fetch an individual value from the server. :c:func:`memcached_mget()` must always be called before using this method. You must pass in a key and its length to fetch the object. You must supply three pointer variables which will give you the state of the returned object. A :c:type:`uint32_t` pointer to contain whatever flags you stored with the value, a :c:type:`size_t` pointer which will be filled with size of of the object, and a :c:type:`memcached_return_t` pointer to hold any error. The object will be returned upon success and NULL will be returned on failure. :c:type:`MEMCACHD_END` is returned by the \*error value when all objects that have been found are returned. The final value upon :c:type:`MEMCACHED_END` is null. -Values returned by :c:func:`memcached_fetch()` must be freed by the caller. -:c:func:`memcached_fetch()` will be DEPRECATED in the near future, -c:func:`memcached_fetch_result()` should be used instead. :c:func:`memcached_fetch_result()` is used to return a :c:type:`memcached_result_st` structure from a memcached server. The result object is forward compatible with changes to the server. For more information please refer to the @@ -105,8 +99,9 @@ RETURN ------ -All objects returned must be freed by the calling application. -:c:func:`memcached_get()` and :c:func:`memcached_fetch()` will return NULL on +All objects retrieved via :c:func:`memcached_get()` or :c:func:`memcached_get_by_key()` must be freed with :manpage:`free(3)`. + +:c:func:`memcached_get()` will return NULL on error. You must look at the value of error to determine what the actual error was. @@ -114,15 +109,10 @@ was. all keys were successful. :c:type:`MEMCACHED_NOTFOUND` will be return if no keys at all were found. -:c:func:`memcached_fetch()` and :c:func:`memcached_fetch_result()` set error +:c:func:`memcached_fetch_result()` sets error to :c:type:`MEMCACHED_END` upon successful conclusion. :c:type:`MEMCACHED_NOTFOUND` will be return if no keys at all were found. -:c:type:`MEMCACHED_KEY_TOO_BIG` is set to error whenever :c::func:`memcached_fetch()` was used -and the key was set larger then :c:type:`MEMCACHED_MAX_KEY`, which was the largest -key allowed for the original memcached ascii server. - - ---- HOME ---- @@ -132,7 +122,6 @@ To find out more information please check: `http://libmemcached.org/ `_ - -------- SEE ALSO -------- diff --git a/docs/memcached_pool.rst b/docs/memcached_pool.rst index 38d2b3a7..7dcc707a 100644 --- a/docs/memcached_pool.rst +++ b/docs/memcached_pool.rst @@ -2,25 +2,25 @@ Working with memcached pools ============================ -.. index:: object: memcached_pool_st - -------- SYNOPSIS -------- #include -.. c:function:: memcached_pool_st *memcached_pool(const char *option_string, size_t option_string_length); +.. c:type:: memcached_pool_st -.. c:function:: memcached_pool_st * memcached_pool_create(memcached_st* mmc, int initial, int max); DEPRECATED - -.. c:function:: memcached_st * memcached_pool_destroy(memcached_pool_st* pool); +.. c:function:: memcached_pool_st* memcached_pool(const char *option_string, size_t option_string_length) + +.. c:function:: memcached_pool_st* memcached_pool_create(memcached_st* mmc, int initial, int max) +.. deprecated:: 0.46 + Use :c:func:`memcached_pool()` instead. -.. c:function:: memcached_st * memcached_pool_pop (memcached_pool_st* pool, bool block, memcached_return_t *rc); +.. c:function:: memcached_st* memcached_pool_destroy(memcached_pool_st* pool) -.. c:function:: memcached_return_t memcached_pool_push(memcached_pool_st* pool, memcached_st *mmc); +.. c:function:: memcached_st* memcached_pool_pop (memcached_pool_st* pool, bool block, memcached_return_t *rc) -.. c:function:: memcached_st *memcached_create (memcached_st *ptr); +.. c:function:: memcached_return_t memcached_pool_push(memcached_pool_st* pool, memcached_st *mmc) .. c:function:: memcached_return_t memcached_pool_behavior_set(memcached_pool_st *pool, memcached_behavior_t flag, uint64_t data) @@ -36,19 +36,7 @@ DESCRIPTION :c:func:`memcached_pool()` is used to create a connection pool of objects you may use to remove the overhead of using memcached_clone for short lived -:c:type:`memcached_st` objects. Please see :manpage:`libmemcached_configuration` for details on the format of the configuration string. - -DEPRECATED :c:func:`memcached_pool_create()` is used to create a connection pool of objects you may use to remove the overhead of using -:c:type:`memcached_clone` for short lived :c:type:`memcached_st` objects. -The mmc argument should be an initialised :c:type:`memcached_st` structure, -and a successfull invocation of :c:type:`memcached_pool_create` takes full -ownership of the variable (until it is released by -:c:type:`memcached_pool_destroy`). The :c:type:`initial` argument specifies -the initial size of the connection pool, and the :c:type:`max` argument -specifies the maximum size the connection pool should grow to. Please note -that the library will allocate a fixed size buffer scaled to the max size of -the connection pool, so you should not pass MAXINT or some other large -number here. +:c:type:`memcached_st` objects. Please see :doc:`libmemcached_configuration` for details on the format of the configuration string. :c:func:`memcached_pool_destroy()` is used to destroy the connection pool created with :c:func:`memcached_pool_create()` and release all allocated @@ -62,23 +50,16 @@ to exceed the maximum size. :c:func:`memcached_pool_push()` is used to return a connection structure back to the pool. -:c:func:`memcached_pool_behavior_set()` and :c:func:`memcached_pool_behagior_get()` is used to get/set behavior flags on all connections in the pool. +:c:func:`memcached_pool_behavior_get()` and :c:func:`memcached_pool_behavior_set()` is used to get/set behavior flags on all connections in the pool. ------ RETURN ------ +:c:func:`memcached_pool_destroy()` returns the pointer (and ownership) to the :c:type:`memcached_st` structure used to create the pool. If connections are in use it returns NULL. -:c:func:`memcached_pool_create()` returns a pointer to the newly created -:c:type:`memcached_pool_st` structure. On an allocation failure, it returns -NULL. - -:c:func:`memcached_pool_destroy()` returns the pointer (and ownership) to the -:c:type:`memcached_st` structure used to create the pool. If connections are in -use it returns NULL. - -:c:func:`memcached_pool_pop()` returns a pointer to a :c:type:`memcached_st structure` from the pool (or NULL if an allocation cannot be satisfied). +:c:func:`memcached_pool_pop()` returns a pointer to a :c:type:`memcached_st` structure from the pool (or NULL if an allocation cannot be satisfied). :c:func:`memcached_pool_push()` returns :c:type:`MEMCACHED_SUCCESS` upon success. @@ -108,4 +89,4 @@ Trond Norbye, SEE ALSO -------- -:manpage:`memcached(1)` :manpage:`libmemcached(3)` :manpage:`memcached_strerror(3)` +:manpage:`memcached(1)` :manpage:`libmemcached(3)` :manpage:`memcached_strerror(3)` :manpage:`libmemcached_configuration(3)` diff --git a/docs/memcapable.rst b/docs/memcapable.rst deleted file mode 100644 index 28988046..00000000 --- a/docs/memcapable.rst +++ /dev/null @@ -1,83 +0,0 @@ -======================================================================= -memcapable - Checking a Memcached server capibilities and compatibility -======================================================================= - -.. index:: object: memcached_st - -Check the server for compatibility and capabilities - - --------- -SYNOPSIS --------- - -:program:`memcapable` - -.. program:: memcapable - ------------ -DESCRIPTION ------------ - -:program:`memcapable` connects to the specified memcached server and tries to -determine its capabilities by running the various commands and verifying -the response. - - ------------ -LIMITATIONS ------------ - - -The current version of memcapable will only verify the binary protocol. - - -------- -OPTIONS -------- - - -.. option:: -h hostname - -Specify the hostname to connect to. The default is \ *localhost*\ - -.. option:: -p port - -Specify the port number to connect to. The default is \ *11211*\ - -.. option:: -c - -Generate a coredump when it detects an error from the server. - -.. option:: -v - -Print out the comparison when it detects an error from the server. - -.. option:: -t n - -Set the timeout from an IO operation to/from the server to \ *n*\ seconds. - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - ------- -AUTHOR ------- - - -Trond Norbye, - - --------- -SEE ALSO --------- - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` - diff --git a/docs/memcat.rst b/docs/memcat.rst deleted file mode 100644 index 5c102207..00000000 --- a/docs/memcat.rst +++ /dev/null @@ -1,73 +0,0 @@ -================================= -memcat - "cat" data from a server -================================= - - -Copy a set of keys to stdout - - --------- -SYNOPSIS --------- - - -:program:`memcat` - -.. program:: memcat - - ------------ -DESCRIPTION ------------ - - -:program:`memcat` outputs to stdout the value a single or multiple set of keys -stored in a memcached(1) server. If any key is not found an error is returned. - -It is similar to the standard UNIX cat(1) utility. - - -------- -OPTIONS -------- - - -You can specify servers via the option: - -.. option:: --servers - -or via the environment variable: - -.. envvar:: `MEMCACHED_SERVERS` - -For a full list of operations run the tool with the option: - -.. option:: --help - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - ------- -AUTHOR ------- - - -Brian Aker, - -Mark Atwood - - --------- -SEE ALSO --------- - - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` - diff --git a/docs/memcp.rst b/docs/memcp.rst deleted file mode 100644 index 1ae45114..00000000 --- a/docs/memcp.rst +++ /dev/null @@ -1,75 +0,0 @@ -============================= -memcp - Copy data to a server -============================= - - -Copies files to a collection of memcached servers - - --------- -SYNOPSIS --------- - -:program: `memcp` - -.. program:: memcp - - ------------ -DESCRIPTION ------------ - - -:program:`memcp` copies one or more files into memcached(1) servers. -It is similar to the standard UNIX cp(1) command. - -The key names will be the names of the files, -without any directory path. - - -------- -OPTIONS -------- - - -You can specify servers via the option: - -.. option:: --servers - -or via the environment variable: - -.. envvar:: `MEMCACHED_SERVERS` - -If you do not specify either these, the final value in the command line list is the name of a server(s). - -For a full list of operations run the tool with the option: - -.. option:: --help - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - ------- -AUTHOR ------- - - -Brian Aker, - -Mark Atwood, - - --------- -SEE ALSO --------- - - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` - diff --git a/docs/memdump.rst b/docs/memdump.rst deleted file mode 100644 index ab7bd02e..00000000 --- a/docs/memdump.rst +++ /dev/null @@ -1,58 +0,0 @@ -============================= -memdump - Dumping your server -============================= - - -Dump a list of keys from a server. - - --------- -SYNOPSIS --------- - -:program: `memdump` - -.. program:: memdump - ------------ -DESCRIPTION ------------ - - -:program:`memdump` dumps a list of "keys" from all servers that -it is told to fetch from. Because memcached does not guarentee to -provide all keys it is not possible to get a complete "dump". - - -------- -OPTIONS -------- - - -For a full list of operations run the tool with option: - -.. option:: --help - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - ------- -AUTHOR ------- - - -Brian Aker, - - --------- -SEE ALSO --------- - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/memerror.rst b/docs/memerror.rst deleted file mode 100644 index aa298993..00000000 --- a/docs/memerror.rst +++ /dev/null @@ -1,58 +0,0 @@ -============================================== -memerror - translate an error code to a string -============================================== - - -Translates a memcached error code into a string - - --------- -SYNOPSIS --------- - -:program: `memerror` - -.. program:: memerror - ------------ -DESCRIPTION ------------ - - -:program:`memerror` translates an error code from libmemcached(3) into a human -readable string. - - -------- -OPTIONS -------- - - -For a full list of operations run the tool with option: - -.. option:: --help - - ----- -HOME ----- - - -To find out more infoerroration please check: -`http://libmemcached.org/ `_ - - ------- -AUTHOR ------- - - -Brian Aker, - - --------- -SEE ALSO --------- - - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/memflush.rst b/docs/memflush.rst deleted file mode 100644 index d2cb644f..00000000 --- a/docs/memflush.rst +++ /dev/null @@ -1,67 +0,0 @@ -======================================= -memflush - flush all data from a server -======================================= - - -Reset a server or list of servers - - --------- -SYNOPSIS --------- - -:program: `memflush` - -.. program:: memflush - ------------ -DESCRIPTION ------------ - - -:program:`memflush` resets the contents of memcached(1) servers. -This means that all data in the specified servers will be deleted. - - -------- -OPTIONS -------- - - -You can specify servers via the option: - -.. option:: --servers - -or via the environment variable: - -.. envvar:: `MEMCACHED_SERVERS` - -For a full list of operations run the tool with option: - -.. option:: --help - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - ------- -AUTHOR ------- - - -Brian Aker, - -Mark Atwood - - --------- -SEE ALSO --------- - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/memrm.rst b/docs/memrm.rst deleted file mode 100644 index 7b34b029..00000000 --- a/docs/memrm.rst +++ /dev/null @@ -1,71 +0,0 @@ -================================= -memrm - Remove data from a server -================================= - - -memrm - Remove a key(s) from a collection of memcached servers - - --------- -SYNOPSIS --------- - -:program: `memrm` - -.. program:: memrm - - ------------ -DESCRIPTION ------------ - - -:program:`memrm` removes items, specified by key, from memcached(1) servers. - - -------- -OPTIONS -------- - - -You can specify servers via the option: - -.. option:: --servers - -or via the environment variable: - -.. envvar:: `MEMCACHED_SERVERS` - -For a full list of operations run the tool with the - -.. option:: --help - - - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - ------- -AUTHOR ------- - - -Brian Aker, - -Mark Atwood, - - --------- -SEE ALSO --------- - - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` - diff --git a/docs/memslap.rst b/docs/memslap.rst deleted file mode 100644 index c99abd77..00000000 --- a/docs/memslap.rst +++ /dev/null @@ -1,64 +0,0 @@ -================================================= -memslap - Load testing and benchmarking a server -================================================= - - --------- -SYNOPSIS --------- - - -:program: `memslap` - -.. program:: memslap - ------------ -DESCRIPTION ------------ - - -:program:`memslap` is a load generation and benchmark tool for memcached(1) -servers. It generates configurable workload such as threads, concurrencies, connections, run time, overwrite, miss rate, key size, value size, get/set proportion, expected throughput, and so on. - - -------- -OPTIONS -------- - - -You can specify servers via the option: - -.. option:: --servers - -or via the environment variable: - -.. envvar:: `MEMCACHED_SERVERS` - -For a full list of operations run the tool with: - -.. option:: --help - - ----- -HOME ----- - - -To find out more information please check: -'http://libmemcached.org/ '_ - - ------- -AUTHOR ------- - - -Brian Aker, - - --------- -SEE ALSO --------- - - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` diff --git a/docs/memstat.rst b/docs/memstat.rst deleted file mode 100644 index 550c346f..00000000 --- a/docs/memstat.rst +++ /dev/null @@ -1,61 +0,0 @@ -========================================= -memstat - Gather statistics from a server -========================================= - - -memstat - Display the operating status of a single or group of memcached servers - - --------- -SYNOPSIS --------- - -:program: `memstat` - -.. program:: memstat - ------------ -DESCRIPTION ------------ - - -:program:`memstat` dumps the state of memcached(1) servers. -It displays all data to stdout. - - -------- -OPTIONS -------- - - -You can specify servers via the option: - -.. option:: --servers - -or via the environment variable: - -.. envvar:: 'MEMCACHED_SERVERS, --args' - -which can be used to specify the "argument" sent to the stats command (ie slab, size, items, etc..). - -For a full list of operations run the tool with: - -.. option:: --help - - ----- -HOME ----- - - -To find out more information please check: -`http://libmemcached.org/ `_ - - --------- -SEE ALSO --------- - - -:manpage:`memcached(1)` :manpage:`libmemcached(3)` - diff --git a/libtest/callbacks.h b/libtest/callbacks.h new file mode 100644 index 00000000..ea7ae182 --- /dev/null +++ b/libtest/callbacks.h @@ -0,0 +1,21 @@ +/* uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + +#ifdef __cplusplus +extern "C" { +#endif + +typedef void* (test_callback_create_fn)(enum test_return_t*); +typedef enum test_return_t (test_callback_fn)(void *); +typedef enum test_return_t (test_callback_runner_fn)(test_callback_fn*, void *); +typedef enum test_return_t (test_callback_error_fn)(const enum test_return_t, void *); + +#ifdef __cplusplus +} +#endif + diff --git a/libtest/collection.h b/libtest/collection.h new file mode 100644 index 00000000..e8f8f62c --- /dev/null +++ b/libtest/collection.h @@ -0,0 +1,19 @@ +/* uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + +/** + A structure which describes a collection of test cases. +*/ +struct collection_st { + const char *name; + test_callback_fn *pre; + test_callback_fn *post; + struct test_st *tests; +}; + + diff --git a/libtest/common.h b/libtest/common.h new file mode 100644 index 00000000..9b00f6de --- /dev/null +++ b/libtest/common.h @@ -0,0 +1,50 @@ +/* vim:expandtab:shiftwidth=2:tabstop=2:smarttab: + * + * uTest + * + * Copyright (C) 2011 Data Differential, http://datadifferential.com/ All + * rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following disclaimer + * in the documentation and/or other materials provided with the + * distribution. + * + * * The names of its contributors may not be used to endorse or + * promote products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +/* + Common include file for libmemached +*/ + +#pragma once + +#include + +#include +#include +#include + +#include diff --git a/libtest/core.h b/libtest/core.h new file mode 100644 index 00000000..2b62161b --- /dev/null +++ b/libtest/core.h @@ -0,0 +1,11 @@ +/* + * uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + +LIBTEST_API +void create_core(void); diff --git a/libtest/error.h b/libtest/error.h new file mode 100644 index 00000000..c4a3a702 --- /dev/null +++ b/libtest/error.h @@ -0,0 +1,18 @@ +/* uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + +enum test_return_t { + TEST_SUCCESS= 0, /* Backwards compatibility */ + TEST_FAILURE, + TEST_MEMORY_ALLOCATION_FAILURE, + TEST_SKIPPED, + TEST_FATAL // Collection should not be continued +}; + + +#define test_failed(__test_return_t) ((__test_return_t) != TEST_SUCCESS) diff --git a/libtest/failed.h b/libtest/failed.h index 16d1aecc..a4b2cacc 100644 --- a/libtest/failed.h +++ b/libtest/failed.h @@ -36,7 +36,6 @@ */ #pragma once -#include #ifdef __cplusplus extern "C" { diff --git a/libtest/framework.cc b/libtest/framework.cc new file mode 100644 index 00000000..ff67f20b --- /dev/null +++ b/libtest/framework.cc @@ -0,0 +1,57 @@ +/* uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#include + +test_return_t Framework::destroy(void* arg) +{ + if (_destroy) + { + return _destroy(arg); + } + + return TEST_SUCCESS; +} + +test_return_t Framework::Item::flush(void* arg, test_st* run) +{ + if (run->requires_flush and _flush) + { + return _flush(arg); + } + + return TEST_SUCCESS; +} + +test_return_t Framework::on_error(const test_return_t rc, void* arg) +{ + if (_on_error and test_failed(_on_error(rc, arg))) + { + return TEST_FAILURE; + } + + return TEST_SUCCESS; +} + +test_return_t Framework::startup(void* arg) +{ + if (collection_startup) + { + return collection_startup(arg); + } + + return TEST_SUCCESS; +} + +test_return_t Framework::Item::startup(void* arg) +{ + if (_startup) + { + return _startup(arg); + } + + return TEST_SUCCESS; +} diff --git a/libtest/framework.h b/libtest/framework.h new file mode 100644 index 00000000..7a58cbee --- /dev/null +++ b/libtest/framework.h @@ -0,0 +1,137 @@ +/* uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + +/** + Framework is the structure which is passed to the test implementation to be filled. + This must be implemented in order for the test framework to load the tests. We call + get_world() in order to fill this structure. +*/ + +struct Framework { + collection_st *collections; + + /* These methods are called outside of any collection call. */ + test_callback_create_fn *_create; + test_callback_fn *_destroy; + + void* create(test_return_t* arg) + { + if (_create) + { + return _create(arg); + } + + return NULL; + } + + test_return_t destroy(void*); + + /* This is called a the beginning of any collection run. */ + test_callback_fn *collection_startup; + + test_return_t startup(void*); + + /* This is called a the end of any collection run. */ + test_callback_fn *collection_shutdown; + + test_return_t shutdown(void* arg) + { + if (collection_shutdown) + { + return collection_shutdown(arg); + } + + return TEST_SUCCESS; + } + + /** + These are run before/after the test. If implemented. Their execution is not controlled + by the test. + */ + struct Item { + /* This is called a the beginning of any run. */ + test_callback_fn *_startup; + + test_return_t startup(void*); + + /* + This called on a test if the test requires a flush call (the bool is + from test_st) + */ + test_callback_fn *_flush; + + /* + Run before and after the runnner is executed. + */ + test_callback_fn *pre_run; + test_callback_fn *post_run; + + Item() : + _startup(NULL), + _flush(NULL), + pre_run(NULL), + post_run(NULL) + { } + + test_return_t flush(void* arg, test_st* run); + + void set_pre(test_callback_fn *arg) + { + pre_run= arg; + } + + void set_post(test_callback_fn *arg) + { + pre_run= arg; + } + + test_return_t pre(void *arg) + { + if (pre_run) + { + return pre_run(arg); + } + + return TEST_SUCCESS; + } + + test_return_t post(void *arg) + { + if (post_run) + { + return post_run(arg); + } + + return TEST_SUCCESS; + } + + } item; + + /** + If an error occurs during the test, this is called. + */ + test_callback_error_fn *_on_error; + + test_return_t on_error(const enum test_return_t, void *); + + /** + Runner represents the callers for the tests. If not implemented we will use + a set of default implementations. + */ + Runner *runner; + + Framework(); + + virtual ~Framework() + { } + + Framework(const Framework&); + +private: + Framework& operator=(const Framework&); +}; diff --git a/libtest/get.h b/libtest/get.h new file mode 100644 index 00000000..2cd9dec8 --- /dev/null +++ b/libtest/get.h @@ -0,0 +1,22 @@ +/* + * uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + +#ifdef __cplusplus +extern "C" { +#endif + + /* How we make all of this work :) */ + LIBTEST_API + void get_world(Framework *world); + +#ifdef __cplusplus +} +#endif + + diff --git a/libtest/include.am b/libtest/include.am index a7a795fd..610521a7 100644 --- a/libtest/include.am +++ b/libtest/include.am @@ -16,8 +16,18 @@ CLEANFILES+= \ tests/var/tmp/* noinst_HEADERS+= \ + libtest/callbacks.h \ + libtest/collection.h \ + libtest/common.h \ + libtest/core.h \ + libtest/error.h \ libtest/failed.h \ + libtest/framework.h \ + libtest/get.h \ + libtest/runner.h \ libtest/server.h \ + libtest/stats.h \ + libtest/strerror.h \ libtest/test.h \ libtest/visibility.h @@ -26,8 +36,9 @@ libtest_libserver_la_SOURCES= libtest/server.c noinst_LTLIBRARIES+= libtest/libtest.la libtest_libtest_la_SOURCES=\ + libtest/framework.cc \ libtest/test.cc -libtest_libtest_la_CFLAGS= ${AM_CFLAGS} ${NO_CONVERSION} +libtest_libtest_la_CFLAGS= ${AM_CFLAGS} ${NO_CONVERSION} -DBUILDING_LIBTEST libtest_libtest_la_CPPFLAGS= ${AM_CPPFLAGS} tests/var: tests/var/log tests/var/tmp diff --git a/libtest/runner.h b/libtest/runner.h new file mode 100644 index 00000000..8678c4c0 --- /dev/null +++ b/libtest/runner.h @@ -0,0 +1,19 @@ +/* uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + + +/** + Structure which houses the actual callers for the test cases contained in + the collections. +*/ +struct Runner { + test_callback_runner_fn *pre; + test_callback_runner_fn *run; + test_callback_runner_fn *post; +}; + diff --git a/libtest/stats.h b/libtest/stats.h new file mode 100644 index 00000000..29be06c6 --- /dev/null +++ b/libtest/stats.h @@ -0,0 +1,30 @@ +/* uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once +struct Stats { + int32_t collection_success; + int32_t collection_skipped; + int32_t collection_failed; + int32_t collection_total; + + uint32_t success; + uint32_t skipped; + uint32_t failed; + uint32_t total; + + Stats() : + collection_success(0), + collection_skipped(0), + collection_failed(0), + collection_total(0), + success(0), + skipped(0), + failed(0), + total(0) + { } +}; + diff --git a/libtest/strerror.h b/libtest/strerror.h new file mode 100644 index 00000000..2d6e6536 --- /dev/null +++ b/libtest/strerror.h @@ -0,0 +1,14 @@ +/* + * uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +#pragma once + +/** + @note Friendly print function for errors. +*/ +LIBTEST_API +const char *test_strerror(test_return_t code); diff --git a/libtest/test.cc b/libtest/test.cc index 2c57d206..ee743983 100644 --- a/libtest/test.cc +++ b/libtest/test.cc @@ -8,28 +8,40 @@ */ -#include +#include -#include -#include -#include -#include +#include +#include +#include #include #include #include #include #include -#include +#include #include #include -#include +#include #ifndef __INTEL_COMPILER #pragma GCC diagnostic ignored "-Wold-style-cast" #endif -static void world_stats_print(world_stats_st *stats) +static in_port_t global_port= 0; + +in_port_t default_port() +{ + assert(global_port); + return global_port; +} + +void set_default_port(in_port_t port) +{ + global_port= port; +} + +static void stats_print(Stats *stats) { std::cout << "\tTotal Collections\t\t\t\t" << stats->collection_total << std::endl; std::cout << "\tFailed Collections\t\t\t\t" << stats->collection_failed << std::endl; @@ -58,19 +70,21 @@ const char *test_strerror(test_return_t code) switch (code) { case TEST_SUCCESS: return "ok"; + case TEST_FAILURE: return "failed"; - case TEST_FATAL: - return "failed"; + case TEST_MEMORY_ALLOCATION_FAILURE: return "memory allocation"; + case TEST_SKIPPED: return "skipped"; - case TEST_MAXIMUM_RETURN: - default: - std::cerr << "Unknown return value." << std::endl; - abort(); + + case TEST_FATAL: + break; } + + return "failed"; } void create_core(void) @@ -94,12 +108,14 @@ void create_core(void) static test_return_t _runner_default(test_callback_fn func, void *p) { if (func) + { return func(p); + } return TEST_SUCCESS; } -static world_runner_st defualt_runners= { +static Runner defualt_runners= { _runner_default, _runner_default, _runner_default @@ -112,29 +128,23 @@ static test_return_t _default_callback(void *p) return TEST_SUCCESS; } -static collection_st *init_world(world_st *world) +Framework::Framework() : + collections(NULL), + _create(NULL), + _destroy(NULL), + collection_startup(_default_callback), + collection_shutdown(_default_callback), + _on_error(NULL), + runner(&defualt_runners) { - if (world->runner == NULL) - { - world->runner= &defualt_runners; - } - - if (world->collection_startup == NULL) - world->collection_startup= _default_callback; - - if (world->collection_shutdown == NULL) - world->collection_shutdown= _default_callback; - - return world->collections; } int main(int argc, char *argv[]) { - world_st world; - void *world_ptr; + Framework world; - world_stats_st stats; + Stats stats; get_world(&world); @@ -143,20 +153,11 @@ int main(int argc, char *argv[]) world.runner= &defualt_runners; } - collection_st *collection= init_world(&world); - - if (world.create) + test_return_t error; + void *world_ptr= world.create(&error); + if (test_failed(error)) { - test_return_t error; - world_ptr= world.create(&error); - if (error != TEST_SUCCESS) - { - return EXIT_FAILURE; - } - } - else - { - world_ptr= NULL; + return EXIT_FAILURE; } char *collection_to_run= NULL; @@ -180,7 +181,7 @@ int main(int argc, char *argv[]) wildcard= argv[2]; } - for (collection_st *next= collection; next->name; next++) + for (collection_st *next= world.collections; next->name; next++) { test_return_t collection_rc= TEST_SUCCESS; bool failed= false; @@ -191,7 +192,7 @@ int main(int argc, char *argv[]) stats.collection_total++; - collection_rc= world.collection_startup(world_ptr); + collection_rc= world.startup(world_ptr); if (collection_rc == TEST_SUCCESS and next->pre) { @@ -216,10 +217,7 @@ int main(int argc, char *argv[]) goto cleanup; case TEST_MEMORY_ALLOCATION_FAILURE: - case TEST_MAXIMUM_RETURN: - default: - assert(0); - break; + test_assert(0, "Allocation failure, or unknown return"); } for (test_st *run= next->tests; run->name; run++) @@ -234,53 +232,21 @@ int main(int argc, char *argv[]) std::cerr << "\tTesting " << run->name; - if (world.run_startup) - { - world.run_startup(world_ptr); - } + world.item.startup(world_ptr); - if (run->requires_flush && world.flush) - { - world.flush(world_ptr); - } - - if (world.pre_run) - { - world.pre_run(world_ptr); - } + world.item.flush(world_ptr, run); + world.item.pre(world_ptr); test_return_t return_code; { // Runner Code -#if 0 - if (next->pre and world.runner->pre) - { - return_code= world.runner->pre(next->pre, world_ptr); - - if (return_code != TEST_SUCCESS) - { - goto error; - } - } -#endif - gettimeofday(&start_time, NULL); return_code= world.runner->run(run->test_fn, world_ptr); gettimeofday(&end_time, NULL); load_time= timedif(end_time, start_time); - -#if 0 - if (next->post && world.runner->post) - { - (void) world.runner->post(next->post, world_ptr); - } -#endif } - if (world.post_run) - { - world.post_run(world_ptr); - } + world.item.post(world_ptr); stats.total++; @@ -305,22 +271,15 @@ int main(int argc, char *argv[]) break; case TEST_MEMORY_ALLOCATION_FAILURE: - case TEST_MAXIMUM_RETURN: - default: - break; - abort(); + test_assert(0, "Memory Allocation Error"); } std::cerr << "[ " << test_strerror(return_code) << " ]" << std::endl; - if (world.on_error) + if (test_failed(world.on_error(return_code, world_ptr))) { - test_return_t rc= world.on_error(return_code, world_ptr); - - if (rc != TEST_SUCCESS) - break; + break; } - } if (next->post && world.runner->post) @@ -334,10 +293,7 @@ int main(int argc, char *argv[]) } cleanup: - if (world.collection_shutdown) - { - world.collection_shutdown(world_ptr); - } + world.shutdown(world_ptr); } if (stats.collection_failed || stats.collection_skipped) @@ -352,18 +308,12 @@ cleanup: std::cout << std::endl << std::endl << "All tests completed successfully." << std::endl << std::endl; } - if (world.destroy) + if (test_failed(world.destroy(world_ptr))) { - test_return_t error= world.destroy(world_ptr); - - if (error != TEST_SUCCESS) - { - std::cerr << "Failure during shutdown." << std::endl; - stats.failed++; // We do this to make our exit code return EXIT_FAILURE - } + stats.failed++; // We do this to make our exit code return EXIT_FAILURE } - world_stats_print(&stats); + stats_print(&stats); return stats.failed == 0 ? 0 : 1; } diff --git a/libtest/test.h b/libtest/test.h index d6d7d787..c36572ea 100644 --- a/libtest/test.h +++ b/libtest/test.h @@ -1,46 +1,12 @@ -/* uTest - * Copyright (C) 2011 Data Differential, http://datadifferential.com/ - * Copyright (C) 2006-2009 Brian Aker - * All rights reserved. +/* + * uTest Copyright (C) 2011 Data Differential, http://datadifferential.com/ * * Use and distribution licensed under the BSD license. See * the COPYING file in the parent directory for full text. */ -/* - Structures for generic tests. -*/ - -#include -#include -#include -#include -#include - #pragma once -enum test_return_t { - TEST_SUCCESS= 0, /* Backwards compatibility */ - TEST_FAILURE, - TEST_MEMORY_ALLOCATION_FAILURE, - TEST_SKIPPED, - TEST_FATAL, // Collection should not be continued - TEST_MAXIMUM_RETURN /* Always add new error code before */ -}; - -#ifdef __cplusplus -extern "C" { -#endif - -typedef void* (test_callback_create_fn)(enum test_return_t*); -typedef enum test_return_t (test_callback_fn)(void *); -typedef enum test_return_t (test_callback_runner_fn)(test_callback_fn*, void *); -typedef enum test_return_t (test_callback_error_fn)(const enum test_return_t, void *); - -#ifdef __cplusplus -} -#endif - /** A structure describing the test case. */ @@ -50,142 +16,10 @@ struct test_st { test_callback_fn *test_fn; }; - -/** - A structure which describes a collection of test cases. -*/ -struct collection_st { - const char *name; - test_callback_fn *pre; - test_callback_fn *post; - struct test_st *tests; -}; - - -/** - Structure which houses the actual callers for the test cases contained in - the collections. -*/ -struct world_runner_st { - test_callback_runner_fn *pre; - test_callback_runner_fn *run; - test_callback_runner_fn *post; -}; - - -/** - world_st is the structure which is passed to the test implementation to be filled. - This must be implemented in order for the test framework to load the tests. We call - get_world() in order to fill this structure. -*/ - -struct world_st { - collection_st *collections; - - /* These methods are called outside of any collection call. */ - test_callback_create_fn *create; - test_callback_fn *destroy; - - /* This is called a the beginning of any collection run. */ - test_callback_fn *collection_startup; - - /* This is called a the end of any collection run. */ - test_callback_fn *collection_shutdown; - - /* This is called a the beginning of any run. */ - test_callback_fn *run_startup; - - /* This called on a test if the test requires a flush call (the bool is from test_st) */ - test_callback_fn *flush; - - /** - These are run before/after the test. If implemented. Their execution is not controlled - by the test. - */ - test_callback_fn *pre_run; - test_callback_fn *post_run; - - /** - If an error occurs during the test, this is called. - */ - test_callback_error_fn *on_error; - - /** - Runner represents the callers for the tests. If not implemented we will use - a set of default implementations. - */ - world_runner_st *runner; - - world_st() : - collections(NULL), - create(NULL), - destroy(NULL), - collection_startup(NULL), - collection_shutdown(NULL), - run_startup(NULL), - flush(NULL), - pre_run(NULL), - post_run(NULL), - on_error(NULL), - runner(NULL) - { } - - virtual ~world_st() - { } - -private: - world_st(const world_st&); - world_st& operator=(const world_st&); -}; - - - -/** - @note world_stats_st is a simple structure for tracking test successes. -*/ -struct world_stats_st { - int32_t collection_success; - int32_t collection_skipped; - int32_t collection_failed; - int32_t collection_total; - - uint32_t success; - uint32_t skipped; - uint32_t failed; - uint32_t total; - - world_stats_st() : - collection_success(0), - collection_skipped(0), - collection_failed(0), - collection_total(0), - success(0), - skipped(0), - failed(0), - total(0) - { } -}; - #define TEST_STRINGIFY(x) #x #define TEST_TOSTRING(x) TEST_STRINGIFY(x) #define TEST_AT __FILE__ ":" TEST_TOSTRING(__LINE__) -#ifdef __cplusplus -extern "C" { -#endif - -/* How we make all of this work :) */ -LIBTEST_API -void get_world(world_st *world); - -LIBTEST_INTERNAL_API -void create_core(void); - -/** - @note Friendly print function for errors. -*/ -LIBTEST_INTERNAL_API -const char *test_strerror(test_return_t code); #define test_assert_errno(A) \ do \ @@ -199,6 +33,17 @@ do \ } \ } while (0) +#define test_assert(A, B) \ +do \ +{ \ + if ((A)) { \ + fprintf(stderr, "\nAssertion, %s(%s), failed at %s:%d: ", (B), #A, __FILE__, __LINE__);\ + fprintf(stderr, "\n"); \ + create_core(); \ + assert((A)); \ + } \ +} while (0) + #define test_truth(A) \ do \ { \ @@ -315,14 +160,3 @@ do \ } \ } while (0) -#ifdef __cplusplus -} -#endif - -#ifdef __cplusplus -#define test_literal_param(X) (X), (static_cast((sizeof(X) - 1))) -#else -#define test_literal_param(X) (X), ((size_t)((sizeof(X) - 1))) -#endif - -#define test_string_make_from_cstr(X) (X), ((X) ? strlen(X) : 0) diff --git a/libtest/test.hpp b/libtest/test.hpp new file mode 100644 index 00000000..5953f82c --- /dev/null +++ b/libtest/test.hpp @@ -0,0 +1,46 @@ +/* uTest + * Copyright (C) 2011 Data Differential, http://datadifferential.com/ + * Copyright (C) 2006-2009 Brian Aker + * All rights reserved. + * + * Use and distribution licensed under the BSD license. See + * the COPYING file in the parent directory for full text. + */ + +/* + Structures for generic tests. +*/ + +#include +#include +#include +#include + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#pragma once + +LIBTEST_API + in_port_t default_port(); + +LIBTEST_API + void set_default_port(in_port_t port); + +#ifdef __cplusplus +#define test_literal_param(X) (X), (static_cast((sizeof(X) - 1))) +#else +#define test_literal_param(X) (X), ((size_t)((sizeof(X) - 1))) +#endif + +#define test_string_make_from_cstr(X) (X), ((X) ? strlen(X) : 0) diff --git a/tests/atomsmasher.cc b/tests/atomsmasher.cc index ffde3877..a525b95e 100644 --- a/tests/atomsmasher.cc +++ b/tests/atomsmasher.cc @@ -30,7 +30,7 @@ #include #include -#include +#include /* Number of items generated for tests */ #define GLOBAL_COUNT 100000 @@ -276,18 +276,18 @@ collection_st collection[] ={ #include "libmemcached_world.h" -void get_world(world_st *world) +void get_world(Framework *world) { world->collections= collection; - world->create= (test_callback_create_fn*)world_create; - world->destroy= (test_callback_fn*)world_destroy; + world->_create= (test_callback_create_fn*)world_create; + world->_destroy= (test_callback_fn*)world_destroy; - world->run_startup= (test_callback_fn*)world_test_startup; - world->flush= (test_callback_fn*)world_flush; - world->pre_run= (test_callback_fn*)world_pre_run; - world->post_run= (test_callback_fn*)world_post_run; - world->on_error= (test_callback_error_fn*)world_on_error; + world->item._startup= (test_callback_fn*)world_test_startup; + world->item._flush= (test_callback_fn*)world_flush; + world->item.set_pre((test_callback_fn*)world_pre_run); + world->item.set_post((test_callback_fn*)world_post_run); + world->_on_error= (test_callback_error_fn*)world_on_error; world->collection_startup= (test_callback_fn*)world_container_startup; world->collection_shutdown= (test_callback_fn*)world_container_shutdown; diff --git a/tests/basic.cc b/tests/basic.cc index 85e96591..8f57e4c5 100644 --- a/tests/basic.cc +++ b/tests/basic.cc @@ -37,7 +37,9 @@ #include #include -#include + +#include + #include "tests/basic.h" #include diff --git a/tests/deprecated.cc b/tests/deprecated.cc index 03cfdaaa..eebc7d06 100644 --- a/tests/deprecated.cc +++ b/tests/deprecated.cc @@ -39,7 +39,7 @@ #include -#include +#include #include #include diff --git a/tests/error_conditions.cc b/tests/error_conditions.cc index 50a64da5..09a01eb4 100644 --- a/tests/error_conditions.cc +++ b/tests/error_conditions.cc @@ -41,7 +41,9 @@ #include #include -#include + +#include + #include test_return_t memcached_increment_MEMCACHED_NO_SERVERS(memcached_st *) diff --git a/tests/error_conditions.h b/tests/error_conditions.h index 1bfa9ed2..54991973 100644 --- a/tests/error_conditions.h +++ b/tests/error_conditions.h @@ -37,8 +37,6 @@ #pragma once -#include - #ifdef __cplusplus extern "C" { #endif diff --git a/tests/hash_plus.cc b/tests/hash_plus.cc index 987d7971..e3130786 100644 --- a/tests/hash_plus.cc +++ b/tests/hash_plus.cc @@ -4,7 +4,7 @@ #include -#include +#include #include #include @@ -93,7 +93,7 @@ static test_return_t set_function_test(void *) { hashkit_return_t rc= hashk.set_function(*algo); - test_true(rc == HASHKIT_SUCCESS); + test_compare(HASHKIT_SUCCESS, rc); uint32_t *list; switch (*algo) @@ -101,30 +101,38 @@ static test_return_t set_function_test(void *) case HASHKIT_HASH_DEFAULT: list= one_at_a_time_values; break; + case HASHKIT_HASH_MD5: list= md5_values; break; + case HASHKIT_HASH_CRC: list= crc_values; break; + case HASHKIT_HASH_FNV1_64: list= fnv1_64_values; break; + case HASHKIT_HASH_FNV1A_64: list= fnv1a_64_values; break; + case HASHKIT_HASH_FNV1_32: list= fnv1_32_values; break; + case HASHKIT_HASH_FNV1A_32: list= fnv1a_32_values; break; + case HASHKIT_HASH_HSIEH: #ifndef HAVE_HSIEH_HASH continue; #endif list= hsieh_values; break; + case HASHKIT_HASH_MURMUR: #ifdef WORDS_BIGENDIAN continue; @@ -134,9 +142,11 @@ static test_return_t set_function_test(void *) #endif list= murmur_values; break; + case HASHKIT_HASH_JENKINS: list= jenkins_values; break; + case HASHKIT_HASH_CUSTOM: case HASHKIT_HASH_MAX: default: @@ -211,7 +221,7 @@ collection_st collection[] ={ {0, 0, 0, 0} }; -void get_world(world_st *world) +void get_world(Framework *world) { world->collections= collection; } diff --git a/tests/hashkit_functions.cc b/tests/hashkit_functions.cc index 461faaed..98396df5 100644 --- a/tests/hashkit_functions.cc +++ b/tests/hashkit_functions.cc @@ -44,7 +44,7 @@ #include -#include +#include #include "hash_results.h" @@ -613,9 +613,9 @@ test_return_t world_destroy(hashkit_st *hashk) return TEST_SUCCESS; } -void get_world(world_st *world) +void get_world(Framework *world) { world->collections= collection; - world->create= (test_callback_create_fn*)world_create; - world->destroy= (test_callback_fn*)world_destroy; + world->_create= (test_callback_create_fn*)world_create; + world->_destroy= (test_callback_fn*)world_destroy; } diff --git a/tests/libmemcached_world.h b/tests/libmemcached_world.h index 36136c6e..fc0b6ae8 100644 --- a/tests/libmemcached_world.h +++ b/tests/libmemcached_world.h @@ -188,7 +188,7 @@ static test_return_t _post_runner_default(libmemcached_test_callback_fn func, li #ifdef __cplusplus -static world_runner_st defualt_libmemcached_runner= { +static Runner defualt_libmemcached_runner= { reinterpret_cast(_pre_runner_default), reinterpret_cast(_runner_default), reinterpret_cast(_post_runner_default) @@ -196,7 +196,7 @@ static world_runner_st defualt_libmemcached_runner= { #else -static world_runner_st defualt_libmemcached_runner= { +static Runner defualt_libmemcached_runner= { (test_callback_runner_fn)_pre_runner_default, (test_callback_runner_fn)_runner_default, (test_callback_runner_fn)_post_runner_default diff --git a/tests/mem_functions.cc b/tests/mem_functions.cc index bc29f2c0..c1bb70fe 100644 --- a/tests/mem_functions.cc +++ b/tests/mem_functions.cc @@ -63,7 +63,7 @@ #define SMALL_STRING_LEN 1024 -#include +#include #include "tests/deprecated.h" #include "tests/parser.h" #include "tests/pool.h" @@ -6507,18 +6507,18 @@ collection_st collection[] ={ #include "tests/libmemcached_world.h" -void get_world(world_st *world) +void get_world(Framework *world) { world->collections= collection; - world->create= (test_callback_create_fn*)world_create; - world->destroy= (test_callback_fn*)world_destroy; + world->_create= (test_callback_create_fn*)world_create; + world->_destroy= (test_callback_fn*)world_destroy; - world->run_startup= (test_callback_fn*)world_test_startup; - world->flush= (test_callback_fn*)world_flush; - world->pre_run= (test_callback_fn*)world_pre_run; - world->post_run= (test_callback_fn*)world_post_run; - world->on_error= (test_callback_error_fn*)world_on_error; + world->item._startup= (test_callback_fn*)world_test_startup; + world->item._flush= (test_callback_fn*)world_flush; + world->item.set_pre((test_callback_fn*)world_pre_run); + world->item.set_post((test_callback_fn*)world_post_run); + world->_on_error= (test_callback_error_fn*)world_on_error; world->collection_startup= (test_callback_fn*)world_container_startup; world->collection_shutdown= (test_callback_fn*)world_container_shutdown; diff --git a/tests/mem_udp.cc b/tests/mem_udp.cc index 650134ad..8153e8ef 100644 --- a/tests/mem_udp.cc +++ b/tests/mem_udp.cc @@ -27,7 +27,7 @@ #include #include -#include +#include #define SERVERS_TO_CREATE 5 @@ -491,18 +491,18 @@ collection_st collection[] ={ #include "libmemcached_world.h" -void get_world(world_st *world) +void get_world(Framework *world) { world->collections= collection; - world->create= (test_callback_create_fn*)world_create; - world->destroy= (test_callback_fn*)world_destroy; + world->_create= (test_callback_create_fn*)world_create; + world->_destroy= (test_callback_fn*)world_destroy; - world->run_startup= (test_callback_fn*)world_test_startup; - world->flush= (test_callback_fn*)world_flush; - world->pre_run= (test_callback_fn*)world_pre_run; - world->post_run= (test_callback_fn*)world_post_run; - world->on_error= (test_callback_error_fn*)world_on_error; + world->item._startup= (test_callback_fn*)world_test_startup; + world->item._flush= (test_callback_fn*)world_flush; + world->item.set_pre((test_callback_fn*)world_pre_run); + world->item.set_post((test_callback_fn*)world_post_run); + world->_on_error= (test_callback_error_fn*)world_on_error; world->collection_startup= (test_callback_fn*)world_container_startup; world->collection_shutdown= (test_callback_fn*)world_container_shutdown; diff --git a/tests/parser.cc b/tests/parser.cc index e18fd01c..ad602048 100644 --- a/tests/parser.cc +++ b/tests/parser.cc @@ -43,13 +43,15 @@ #include #include +#include + #define BUILDING_LIBMEMCACHED // !NEVER use common.h, always use memcached.h in your own apps #include #include -#include "tests/parser.h" -#include "tests/print.h" +#include +#include enum scanner_type_t { diff --git a/tests/parser.h b/tests/parser.h index 15d57a64..dac1a691 100644 --- a/tests/parser.h +++ b/tests/parser.h @@ -37,8 +37,6 @@ #pragma once -#include - #ifdef __cplusplus extern "C" { #endif diff --git a/tests/plus.cpp b/tests/plus.cpp index efb705a7..f67fcf27 100644 --- a/tests/plus.cpp +++ b/tests/plus.cpp @@ -14,7 +14,7 @@ #include -#include +#include #include #include @@ -220,18 +220,18 @@ collection_st collection[] ={ #include "libmemcached_world.h" -void get_world(world_st *world) +void get_world(Framework *world) { world->collections= collection; - world->create= reinterpret_cast(world_create); - world->destroy= reinterpret_cast(world_destroy); + world->_create= reinterpret_cast(world_create); + world->_destroy= reinterpret_cast(world_destroy); - world->run_startup= reinterpret_cast(world_test_startup); - world->flush= reinterpret_cast(world_flush); - world->pre_run= reinterpret_cast(world_pre_run); - world->post_run= reinterpret_cast(world_post_run); - world->on_error= reinterpret_cast(world_on_error); + world->item._startup= reinterpret_cast(world_test_startup); + world->item._flush= reinterpret_cast(world_flush); + world->item.set_pre(reinterpret_cast(world_pre_run)); + world->item.set_post(reinterpret_cast(world_post_run)); + world->_on_error= reinterpret_cast(world_on_error); world->collection_startup= reinterpret_cast(world_container_startup); world->collection_shutdown= reinterpret_cast(world_container_shutdown); diff --git a/tests/pool.cc b/tests/pool.cc index ed5016f0..9e1d94f7 100644 --- a/tests/pool.cc +++ b/tests/pool.cc @@ -37,10 +37,12 @@ #include +#include + #include #include #include -#include +#include #include #include diff --git a/tests/pool.h b/tests/pool.h index 45bb14da..ff4d47c7 100644 --- a/tests/pool.h +++ b/tests/pool.h @@ -37,8 +37,6 @@ #pragma once -#include - #ifdef __cplusplus extern "C" { #endif diff --git a/tests/print.cc b/tests/print.cc index 570ab0f0..b3c8d066 100644 --- a/tests/print.cc +++ b/tests/print.cc @@ -40,7 +40,7 @@ #include #include -#include +#include #include "tests/print.h" diff --git a/tests/replication.cc b/tests/replication.cc index 92d68066..3e4476c5 100644 --- a/tests/replication.cc +++ b/tests/replication.cc @@ -37,7 +37,7 @@ #include -#include +#include #include "libmemcached/common.h" #include diff --git a/tests/string.cc b/tests/string.cc index 5c6175f8..4048d91c 100644 --- a/tests/string.cc +++ b/tests/string.cc @@ -38,6 +38,9 @@ #define BUILDING_LIBMEMCACHED #include + +#include + #include #include diff --git a/tests/string.h b/tests/string.h index eebd0c11..12f06109 100644 --- a/tests/string.h +++ b/tests/string.h @@ -37,8 +37,6 @@ #pragma once -#include - #ifdef __cplusplus extern "C" { #endif diff --git a/tests/virtual_buckets.cc b/tests/virtual_buckets.cc index 308fa1d8..1714600e 100644 --- a/tests/virtual_buckets.cc +++ b/tests/virtual_buckets.cc @@ -37,6 +37,7 @@ #include +#include #include #include diff --git a/tests/virtual_buckets.h b/tests/virtual_buckets.h index 054ec492..dee47694 100644 --- a/tests/virtual_buckets.h +++ b/tests/virtual_buckets.h @@ -37,8 +37,6 @@ #pragma once -#include - struct memcached_st; #ifdef __cplusplus