ChangeLog of IP Virtual Server module for Linux kernel 2.5 ========================================================== IP Virtual Server Netfilter module - Version 1.1.7 - July 5, 2003 Changes: * Split the ip_vs_proto_ahesp.c into ip_vs_proto_ah.c and ip_vs_proto_esp.c. * Julian and Wensong added the choice of more hash functions for IPVS connection entries, such as SHIFTXOR, GOLDENRATIO and JENKINS hash functions. And, the IPVS connection random value is introduced in these hash functions, which can prevent from being attacked against the hash functions. * Changed the maximum value of connection threshold from 65535 to INT_MAX (2147483647). * Changed the default syncid from 255 to 0. The syncid zero at backup daemon means no filtering at all, but accepts all the connection synchronization messages. * Updated the ip_vs_sync.c to the Linux kernel 2.5.74. * Some tidy up ipvsadm. IP Virtual Server Netfilter module - Version 1.1.6 - June 8, 2003 Changes: * Fixed the ip_vs_nq.c to display the loh value in the debugging information correctly. * Initialized the slow timer indexes correctly, otherwise it would cause a big delay for connection expiration, which might lead to some serious problems, such as resource wasting and hang up while unloading the ipvs module. Thanks must go to Horms for the finding and explanation. * Alexandre Cassen added the support of starting both master and backup sync daemons at a time, and added the SyncID support for filtering incoming sync messages. * Changed the way to get the information about the master and backup sync daemons nicely. * Changed the Kconfig and Makefile to configure IPVS transport protocol load balancing support, such as TCP, UDP, ESP and AH. * Adapted the linux_ip_fw_compat_c.diff to the kernel 2.5.70. * ipvsadm update to 1.24-1 - Alexandre Cassen added the SyncdID support to filter incoming sync messages. - Changed libipvs to stop the sync daemon according to the state, and get the information about both master and backup daemon. - Added the description of the --syncid opttion int ipvsadm man page. IP Virtual Server Netfilter module - Version 1.1.5 - May 10, 2003 Changes: * Added the Shortest Expected Delay scheduling module. * Added the Never Queue scheduling module. Thanks must go to Marko Buuri for talking the SED and NQ algorithms to us. * Adapted the initialization of struct nf_hook_ops variables to the Linux kernel 2.5.69. * ipvsadm man page update Added the description of the SED and NQ schedulers. Some documenatation tidy up. IP Virtual Server Netfilter module - Version 1.1.4 - April 18, 2003 Changes: * Updated ipvsadm to version 1.23, so that it can display the counter of persistent connections by "ipvsadm -Ln --persistent-conn", and updated the ipvsadm.8 man page for the description of the "--persistent-conn" option. * Changed the libipvs library to only work with ipvs-1.1.4 or above. * Added the counter of persistent connections per real server. * Changed the overhead ratio of active/inactive connections from 50 to 256 in the least-connection scheduler too. * Removed the unnecessary Docdir setting in the ipvsadm.spec.in. IP Virtual Server Netfilter module - Version 1.1.3 - March 22, 2003 Changes: * Added the ip_vs_use_count_inc/ip_vs_use_count_dec, so that it is easy to adapt the code to the module handling change in the Linux kernel 2.5.xx. * Updated the module use count handling code to the kernel 2.5.65. * Updated the ipvs sync daemon code to the kernel 2.5.65. * Changed the overhead ratio of active/inactive connections from 50 to 256 in the weighted least-connection scheduler, because too small ratio number may lead to load imbalance under highly loaded sites. * Updated the Makefile * Tidy up some error message dispaly in the ipvsadm.c and fixed the Makefile of ipvsadm to work with rpmbuild. IP Virtual Server Netfilter module - Version 1.1.2 - December 19, 2002 Changes: * Changed to set the maximum length of connection synchronization message according to the MTU of specified multicast interface. * Changed the sysctl variable sync_threshold to have two values. The first value is the minimum number of packets that a connection needs to receive before it will be synchronized. The second value is the number of packets received each time to synchronize a connection. To be more accurate, when the condition (received_packet_number % sync_threshold[1] == sync_threshold[0]) is satisfied, the connection will be synchronized right now. * Updated Makefile for the Linux kernel 2.5.52. Note: the module-init-tools-0.9.4.tar.gz or later is needed to load kernel modules. IP Virtual Server Netfilter module - Version 1.1.1 - November 30, 2002 Changes: * Corrected the inc->usecnt updating in the ip_vs_app_inc_put. * Fixed the bugs in the do_ip_vs_get_ctl() that return immediately without releasing __ip_vs_mutex. * Updated Makefile for the kernel 2.5.49. * Changed to use kconfig to configure IPVS, when IPVS code is copied to kernel source. Thank Jan Holmberg for some help on this. * ipvsadm updates o Ported the --sort option of ipvsadm-1.21 to ipvsadm-1.22. o Added the missed ipvs_set_timeout function into libipvs. o Adapted the modprobe_ipvs to use new modprobe for kernel 2.5.49 Note: module-init-tools-0.7.tar.gz or later is needed to load kernel modules since kernel 2.5.48. * ip_vs_ctl.c (ip_vs_add_service): fixed NULL pointer dereferencing when the specified scheduler is not found. * ip_vs_xmit.c (IP_VS_XMIT): added to use dst_output to send skb at the NF_LOCAL_OUT hook, instead of sending it out directly by ip_send(). * Added the standalone struct sltimer_list for slow timer in IPVS, so that IPVS slow timers for connections can work correctly under new scalable SMP per-CPU timer handling in kernel 2.5. IP Virtual Server Netfilter module - Version 1.1.0 - September 15, 2002 Changes: * Julian added new protocol load balancing support model, and added AH/ESP load balancing support for IPVS, which can be used to load balancing IPSec applications. * Revised all the data structures exchanged between IPVS kernel module and user-space program, in order to make it simple and efficient to exchange data. * Added the connection threshold support, where many ideas were borrowed from Ratz's Threshhold patch. Tuned the SH and DH schedulers to use the connection threshold to check if a server is overloaded. * Changed ip_vs_sync.c to work on the Linux kernel 2.5.xx. * Julian and I did many other code revision. * Updated ipvsadm to the version 1.22. Changed the libipvs to use the new ipvs data structure exchanged between kernel and user-space. Changed ipvsadm.c to use this new libipvs. Added the thresholds options to manage connection thresholds feature of ipvs inside the kernel. Updated the ipvsadm man page. ChangeLog of IP Virtual Server module for Linux kernel 2.4 ========================================================== IP Virtual Server Netfilter module - Version 1.0.6 - Auguest 17, 2002 Changes: * Updated connection synchronization code again Some changes in 1.0.5 was not correct, so rolled back to assigning the sync_id inside the sync_thread, and the sync_id can be used to detect whether the sync_thread is alive or not. Changed to let the init process be the parent process of sync thread (fork twice), the init process is the only one to reap the sync thread zombie. It can avoid many problems in the different situations. IP Virtual Server Netfilter module - Version 1.0.5 - Auguest 10, 2002 Changes: * updates of ipvs connection synchronization code Fixed the bug that the original waitpid(sync_pid, ...) doesn't work correctly. Thanks go to Alexandre Cassen for reporting the sync daemon zombie problem. Fixed the locking bug in the ipvs sync master daemon, it should disable bh/sofqirq, because it shares data with softirq, and the sync daemon may be interruptted by softirq on the same CPU (The kernel thread can be pre-empted by interrupts and the interrupt handlers may wake up softirq), otherwise it would lock up the system. Thank Bradley McLean for testing the fix. Change the start_sync_thread() to exit after the sync kernel thread is really started up. * ipvsadm updates Horms changed the ipvsadm.sh to force numeric mode when saving ipvsadm setup. Horms changed the ipvsadm-restore to filter out the comments instead of the ipvsadm init script. There is probably a bug in the popt library, because the flag POPT_ARGFLAG_OPTIONAL is not handled correctly. For example, the command "ipvsadm -A -f 1 -p -s wlc" will have problems. The simple fix for popt-1.6.4 is at ipvsadm/contrib/popt-optional.diff. IP Virtual Server Netfilter module - Version 1.0.4 - July 11, 2002 Changes: * Fixed the bug of updating the state of a normal connection incorrectly in connection synchronization. It might lead negative server connection counter problem when a load balancer synchronize a connection to a backup load balancer, later the backup load balancer takes over and synchronize the connection back to the load balancer. Thank Horms for reporting the problem. * Limit the IP_VS_CONN_TAB_BITS in [8, 20], which means size of the IPVS connection hash table can be 2^8, 2^9, .. , 2^20. It can avoid some configuration problems. * Changed all the IPVS time variables exchanged between user-space and kernel-space in the unit of seconds. It can avoid the problems when the HZ macro may be defined different in user-space and kernel-space. * Updated ipvsadm to the version 1.21. IP Virtual Server Netfilter module - Version 1.0.3 - May 30, 2002 Changes: * Fixed the typo (it should be "-f" in the fwmark example) in ipvsadm man page. Thank George L Chamales for pointing it out. * Julian added the checking of outgoing device hardware header length in the DR and NAT transmitters. If the hardware header length is larger than the skb_headroom(skb), then expand the skb data so that it can hold the hardware header. It fixes up the problem in Ethernet & TokenRing mixed cluster reported by Richard.Newman@alltelmd.com. * Julian fixed the locking bug in the ip_vs_stats_get_info(), by changing the spin_[un]lock calls to the spin_[un]lock_bh. This bug may lockup the SMP box, when the "/proc/net/ip_vs_stats" is read and the system is under load at the same time. * Added the skb nonlinear checking and headroom checking for outgoing device hardware header in the ip_vs_bypass_xmit too. IP Virtual Server Netfilter module - Version 1.0.2 - April 3, 2002 Changes: * Fixed the bug of not checking the result of the down_interruptible() call. Thanks must go to Arjan van de Ven , Bryce, and Julian for this fix. IP Virtual Server Netfilter module - Version 1.0.1 - March 25, 2002 Changes: * Ratz fixed up two typos in the comments of Makefile and ip_vs_wlc.c. * Corrected the FILES in the ipvsadm manual and the other comments in the code. * Added the function of using the greatest common divisor of all the server weights as decreasing interval, instead of 1. It can help avoid the (startup) scheduling problem of big server weights. For example, there are two servers A and B, which have weight 300 and 500 respectively. With using 1 as decreasing interval, the scheduler will send B the first 200 connections. With using the gcd (100 here) as the interval, the scheduler will send connections in the sequence BBABABABBBAB... * Added skipping non-first embedded TCP/UDP fragments in the function ip_vs_in_icmp/ip_vs_out_icmp, as Julian suggested. * Julian removed the unnecessary __init/__exit in the declaration of ip_vs_control_init/ip_vs_control_cleanup, which cause the compiling problems with new binutils. * Julian optimize the function gcd() with non-recursive one. IP Virtual Server Netfilter module - Version 1.0.0 - February 27, 2002 Changes: * Removed the use_dst sysctl variable, which was used for testing purposes. IP Virtual Server Netfilter module - Version 0.9.10 - January 31, 2002 Changes: * Julian changed htons(IP_MF|IP_OFFSET) to _constant_htons( IP_MF|IP_OFFSET) * A little bit code tidy up. * Julian and I worked on avoiding the possible race of expire a connection entry among the ip_vs_conn_flush, ip_vs_random_dropentry, and slow timer, though it is hard to hit. Changed ip_vs_conn_expire_now(cp) to let slow timer expire it, disabled BH in the ip_vs_conn_flush(), changed the ip_vs_conn_expire to hash the connection back if it cannot be expired now, etc. * Fixed the negative check bug in the internal_add_sltimer(), so that the connection whose expire is less than slow timer jiffies can be expired quickly. IP Virtual Server Netfilter module - Version 0.9.9 - January 24, 2002 Changes: * Changed to install ipvsadm man pages according to the %{_mandir} * Fixed the comment "cli-ed" in slow timer code, thank Isaac Claymore for pointing it out. * Fixed the null pointer dereference bug in the debugging information output of __ip_vs_get_out_rt() (dest->addr should be cp->daddr), thank Rodger Erickson for finding it out. * Tidy up some code in do_ip_vs_set_ctl and ip_vs_set_state, and tidy up some comments too. IP Virtual Server Netfilter module - Version 0.9.8 - December 11, 2001 Changes: * Added the nat_icmp_send sysctl variable, which is to control sending icmp messages for VS/NAT when the load balancer receives packets from real servers but the connection entries don't exist. It is disabled by default. * Tidy up some code of slow timer, such as changing the type of __ip_vs_sltimerlist_lock from rwlock to spinlock and the type of functions. Thank Julian for reminding me that local_bh_disable call that I added in the 0.9.8pre1 is not necessary, and testing 0.9.8pre2 on SMP box. * Changed to send the member state and flags of connection synchronization in network order, instead of host order. It can solve the problem that the primary load balancer and the backups may have different hardware architectures. Thank Rodger Erickson for the suggestion. * Added the functionality of synchronizing connection controller each time a connection is synchronized and it has its controller. It is useful for persistent service in the IPVS connection synchronization. And, change the type of protocol in struct ip_vs_sync_conn from __u16 to __u8, then a __u8 space is reserved for future development. * Fixed the bug of linking a dest into a dest_list in the lblcr scheduler. IP Virtual Server Netfilter module - Version 0.9.7 - November 23, 2001 Changes: * Changed ip_vs_skb_cow to call pskb_expand_head, instead of pskb_copy, then allocation of a new sk_buff can be avoided, it is faster when sk_buff is cloned. It is only effective for VS/NAT. * Added the macro IP_VS_DBG_RL that calls net_ratelimit() to limit debugging information. It is now used to guard debugging messages about transmission error. And, added the IP_VS_ERR_RL to guard error messages. * Fixed up the SET/GET command overlapping problem in the libipvs, and added more error messages into ipvs_strerror. * Changed all the function calls in the ipvs helpers to pass the pointer of skb, instead of the pointer of pointer of skb, because the skb will never be reallocated in the helpers now. * Tidy up ipvsadm code a little bit. * Julian and I added the sanity check in VS/DR and VS/TUN transmitters to checksum iphdr if the skb is nonlinear. Changed the iphdr checksum update in the ip_vs_in_icmp, ip_vs_out_icmp and ip_vs_forward_icmp. And, Julian has done a lot of testing on fragments for this release. Thank Julian. IP Virtual Server Netfilter module - Version 0.9.6 - November 4, 2001 Changes: * Julian fixed up the locking of __ip_vs_copy_stats by disabling the bottom half, the original simple spin locking may cause the deadlock between the "ipvsadm -L ..." thread and the bottom half thread of ipvs. * Changed to use ip_vs_scheduler_get/ip_vs_scheduler_put for finding and releasing a scheduler, and fixed up segfault in adding a service with a scheduler doesn't exist and cannot be automically loaded. Thank Julian for the finding and his fix. * Added the IP_VS_SO_SET_ZERO setsockopt and its processing code to set the counters zero in a service or all. * Added the temporary solution (bind_mcastif_addr) to fix up that the source address of UDP multicast messages is bound with address of default gateway interface, instead of address of specified multicast interface in the connection synchronization. Thank Fabrice Bucher for reporting the problem. I think that udp_connect in the kernel actually cause the problem above. :) * Julian and I spent a lot of time making the sk_buff handling for VS/NAT fast and correct. The ipvs 0.9.5 and 0.8.2 don't have problems on this issue, becuase netfilter always linearizes every sk_buff received at nf_hook_slow. This sk_buff linearization will be pushed down to each netfiler hook in the kernel soon, because it is slow. We changed ip_vs_header_check to guarantee TCP/UDP header available in skb data area, added skb_linearize calling only when it is nonlinear and it has an application helper, changed ip_vs_skb_cow to call pskb_copy to copy skb data area only if it is cloned, added the similar change to icmp handling, added the ip_defrag() calling to reassemble IP fragments in the ip_vs_forward_icmp code, etc. :) * Rewrote ip_vs_skb_replce without allocating a new sk_buff, it is much faster than the original one, it is more correct, this function is only used in application helper. Tidy up the code of ip_vs_app.c and ip_vs_ftp.c. * Tidy up a lot code in ip_vs_sync.c, such as making error handling correct, and making the variable names and function prototypes look nicer. * ipvsadm updates Added "-Z, --zero" to zero the packet, byte and rate counters of a service or all services. Added more error message handling in the libipvs library and the ipvsadm.c code. ipvsadm man page update. IP Virtual Server Netfilter module - Version 0.9.5 - October 19, 2001 Changes: * Fixed the type of inactconns and activeconns in the struct ip_vs_dest_user (from u_int16_t to u_int32_t). * Fixed the problem in the command "ipvsadm -S" when the real server hostname is longer than 28 characters and/or the virtualserver name is longer than 33 characters. * Rewrote/reorder the code of VS/NAT incoming/outgoing packet processing, such as added ip_defrag() to reassemble fragments in ip_vs_out(), making checksum related code fast and ncie {then csum_partial(buff, 0, unaligned_addr)/i386 can be avoided}, adding TCP/UDP header checking only for VS/NAT, and stripping off the original ip_vs_proto_doff. It is done with a lot of messages exchanged with Julian, thank Julian. * Added the MODULE_LICENSE("GPL") into each module file, thank Ratz. IP Virtual Server Netfilter module - Version 0.9.4 - September 18, 2001 Changes: * Added the rate estimator, then we can see the rate of each destination, virtual services and the whole LVS (in connections/seconds, packets/second, or bytes/second). * Julian added incremental checksum update for VS/NAT. It will greatly improve performance of VS/NAT services. * Julian added strict length checking of ICMP packet in the IPVS icmp handling. * Added a lot of getsockopt options, such as IP_VS_SO_GET_INFO, IP_VS_SO_GET_SERVICES, IP_VS_SO_GET_SERVICE, IP_VS_SO_GET_TIMEOUTS and IP_VS_SO_GET_DAEMON, so that getsockopt can be used to get all kinds of information about LVS info inside the kernel. * Almost rewrite the /proc/net/ip_vs_stats to output the statistics and rate information of the whole IPVS only. * fixed the wrong MOD_DEC_USE_COUNT in the ip_vs_add_service when no memory available. * tidy up a lot code in the ip_vs_ctl.c * ipvsadm update to the version 1.20. Added the libipvs library to manipulate IPVS through [gs]etsockopt. Change ipvsadm to use the libipvs library to interact IPVS inside the kernel. Added a lot of new options, such as --timeout, --daemon, --stats, and --rate. See the "ipvsadm -h" or ipvsadm man page for more information. Tidy up a lot code of ipvsadm.c, such as removing the function of listing the IPVS services through /proc/net/ip_vs, after we use to getsockopt to get all the information of IPVS; and rewriting the process_options and check_ipvs_version. ipvsadm.c is now neat and short. :) IP Virtual Server Netfilter module - Version 0.9.3 - August 27, 2001 Changes: * Julian changed to use OR operation on skb->nfcache for socket buffer belongs to ipvs, instead of assignment operation. Then, it will not conflict with other netfilter modules. * Julian cleanup some macro defines that use "##msg", so that gcc-2.96 can compile without any warnings. * Added the IP_VS_SO_GET_SERVICE and the IP_VS_SO_GET_DESTS options in IPVS getsockopt, so that the getsockopt can be called to get all the information about a virtual service. * Changed the type of __ip_vs_mutex to semaphore, because [gs]etsockopt may sleep. * Disable the multicast loopback in the syncmaster daemon. * Tidy up code, such as adding a macro of ip_vs_conn_expire_now. * ipvsadm updates (1.19-1) List the service only if a service is selected. The command syntax is "ipvsadm -L -t|u|f service-address [-n]". Update ipvsadm man page for the above change. * ipvsadm updates (1.18-2) 2001-06-23 Rewrote most code of parsing options (either in popt version or in getopt_long version) and processing options. Resolved the duplicate short-name options. For example, the -R option is used either for "ipvsadm -R" to restore ipvs rules, or for "-R server-address" to specify real server address; and the -s is used either for "ipvsadm -s tcp tcpfin udp" to set timeouts, or for "-s scheduler" to specify scheduler. Now, the -R is to restore ipvs rules only, and the -r is to specify server-address; the --set is to set timeouts, and the -s is to specify schedulers. Made the save/restore options work both in popt version and getopt_long version. Add some decriptions about starting/stopping sync daemon in ipvsadm help. Keep code neat and shorter (from 1735 lines to 1642) Update ipvsadm man page for these option changes IP Virtual Server Netfilter module - Version 0.9.2 - June 20, 2001 Changes: * Fixed the missing break statement bug in ip_vs_set_state. Thank Herve Masson for reporting it. * Fixed the missing atomic_inc(&cp->in_pkts) bug in ip_vs_in/ip_vs_in_icmp/ip_vs_leave. * Added the feature of connection synchroniztion from the primary load balancer to the backup load balancers through multicast. The ipvs syncmaster daemon is started inside the kernel on the primary load balancers, and it multicasts the queue of connection state that need synchronization. The ipvs syncbackup daemon is started inside the kernel too on the backup load balancers, and it accepts multicast messages and create corresponding connections. * Changed the ip_vs_fwd_tag, the way of binding packet transmitters, the way of ipvs destionation route lookup, and ip_vs_ftp.c, so that they can work well with the synchronized connections. * Tidy up code, such as removing the unnecessary EXPORT_SYMBOL of ip_vs_bind_dest and ip_vs_unbind_dest. * ipvsadm updates fixed the unknown option bug of the first command with long name in the popt version. added the options to start/stop ipvs sync daemon. the version works with ipvs version 0.9.2 or later. IP Virtual Server Netfilter module - Version 0.9.1 - June 4, 2001 Changes: * Removed the MOD_INC_USE_COUNT/MOD_DEC_USE_COUNT in the ip_vs_conn_new/ip_vs_conn_expire, and added some code to flush all the connections in the ip_vs module exit. Now, the ip_vs module can be removed without waiting for all the connections to be expired. * Added the net.ipv4.vs.expire_nodest_conn sysctl variable. The default value is 0, the load balancer will silently drop packets when its destination server is not available. It may be useful, when user-space monitoring program deletes the destination server (because of server overload or wrong detection) and add back the server later, and the connections to the server can continue. If this variable is set 1, the load balancer will expire the connection immediately when a packet arrives and its destination server is not available, then the client program will be notified that the connection is closed. This is equivalent to the feature some people requires to flush connections when its destination is not available. * Removed the extra empty list checking in the lc, wlc, lblc and lblcr schedulers. IP Virtual Server Netfilter module - Version 0.9.0 - May 10, 2001 Changes: * Added svc->usecnt counter, then svc->refcnt is for counting pointer reference, and svc->usecnt is for counting use. The big service lock in statistics updating was removed. * Removed the __ip_vs_set_expire and __ip_vs_unset_expire functions, and changed the __ip_vs_conn_put and ip_vs_conn_put functions. Now, the timer are only modified when it is put back. * Changed the prototype of ip_vs_conn_new, ip_vs_bind_dest function call was moved inside ip_vs_conn_new. It could remove ip_vs_bind_dest function calls at many place and keep the code short. And, the ip_vs_conn_new and ip_vs_conn_expire are symmetric now, looks nicer. * Applied Julian's persistent template patch, i.e. changing the persistent connection template for fwmark-based service from to , so that different fwmark-based services that share the same VIP can work correctly. * Cleanup a lot of unnecessary white spaces, code is now more vi-friendly. :) * Wenzhuo fixed the typo of failure function in ipvsadm.sh IP Virtual Server Netfilter module - Version 0.8.0 - May 9, 2001 Changes: * Removed some unused function prototypes in the ip_vs.h * Removed the use_dst sysctl variable. IP Virtual Server Netfilter module - Version 0.2.12 - May 5, 2001 Changes: * Adapted the ip_select_ident call in the ip_vs_tunnel_xmit to the change of kernel 2.4.4, and set skb->ip_summed CHECKSUM_NONE, which might be useful. IP Virtual Server Netfilter module - Version 0.2.11 - April 20, 2001 Changes: * Fixed an old bug in the ip_vs_unbind_dest. Its atomic_dec_and_test may have race with __ip_vs_del_dest. Changed atomic_dec_and_test to atomic_dec in the ip_vs_unbind_dest, because the destination will be either in a service's destination list or in the destination trash and there is no need to release it here. Thank Julian for pointing it out. * Fixed the possible memory leakage that the destinations in the trash haven't been released after ipvs module exit, ip_vs_trash_cleanup was added to clean up the destinations in the trash before module exit. IP Virtual Server Netfilter module - Version 0.2.10 - April 19, 2001 Changes: * Fixed the possible race that the service is referred in the meanwhile it may be released by ip_vs_flush. The ip_vs_flush need to wait until all the service users leave, before releasing the service. * Fixed the missing __ip_vs_dst_reset bug in the ip_vs_unbind_dest. Before the destination server is released in the ip_vs_unbind_dest, the __refcnt of dst_cache need to be decreased. IP Virtual Server Netfilter module - Version 0.2.9 - April 12, 2001 Changes: * Fixed the negative connection counting problem. The possible race in connection state transition on SMP would lead to the negative counting problem, a spinlock was added to make core state transition atomic. Thank Jason Steenblik for reporting the problem and testing the fix. * Changed to declare the flags, state and timeout members of struct ip_vs_conn volatile. * Changed to binding application helper and setting state of the connection entry before hashing it in the connection table. IP Virtual Server Netfilter module - Version 0.2.8 - April 4, 2001 Changes: * Julian fixed the wrong ip_vs_conn_put in icmp handling code. * Fixed the wrong setting expiration of ipvs connection entries which would lead to the wrong add_sltimer/del_sltimer calls in the SMP environment. * Removed the unnecessary del_sltimer in the ip_vs_sched_persist. * Code clarification changing ip_vs_table to ip_vs_conn_tab. adding "static" to local variables. removing the used variables in the ip_vs.h removing the unnecessary "#define EXPORT_SYMTAB" in the code. * Julian added the NO_CPORT checking in the ip_vs_random_dropentry and the ip_vs_unbind_dest. * Julian and Wensong revisited ip_vs_conn_put, __ip_vs_conn_put and ip_vs_conn_expire in the SMP environment, hope the the possible race has been removed. Thank Jason Steenblik for some testings. IP Virtual Server Netfilter module - Version 0.2.7 - March 22, 2001 Changes: * Removing the unnecessary strict buffer boundary checking, because the kernel proc buffer already leave some slack (1024 bytes) for overruns. Thank Julian for pointing it out for me. * Changed to use 64-bit variables for byte counters instead of 32-bit. Thank Julian and Ratz for suggesting this feature and the help. * Fixed the wrong printing in /proc/net/ip_vs, which was introduced in 0.2.6, and removed all the compiling warnings when ipvs is built in the kernel. Thank Julian and Ratz for the help too. * Added many checking that if schedulers or application helpers are not modules, don't increase/decrease their use_count. Thank Bruno Cornec for reporting the kernel crashs when everything is built into kernel (no modules). * Clarified some declarations of ip_vs_stats and __ip_vs_svc_lock so that ipvs can be built into kernel with the SMP option, and claried some inconsistent declarations of fwmark too. * ipvsadm updates fixed some warnings while compiling config_stream.c and dynamic_array.c. fixed the error result of "ipvsadm -R" and "ipvsadm -S". added ipvsadm-save and ipvsadm-restore man pages. added the "-n" option into ipvsadm-save. changed Makefile so that it can install ipvsadm.sh, ipvsadm-save and ipvsadm-restore to the right places. added trying to request the ip_vs module when the ipvs service isn't available. Thank Joe for suggesting this feature. IP Virtual Server Netfilter module - Version 0.2.6 - March 18, 2001 Changes: * ipvsadm updates Horms fixed that ipvsadm cannot read /etc/sysconfig/ipvsadm correctly on Red Hat 7. It is caused by scanf under the glibc that ships with Red Hat 7 not returning EOF as the man page documents it will once input stops. Added some code in the config_stream.c so that it can detect ab EOF. Horms tidys up ipvsadm-restore, ipvsadm-save and ipvsadm.sh a little too. * Julian fixed the dst_cache usage, i.e. __ip_vs_dst_check() needs doing dst_hold(dst) before returning an available dst. Then, the __refcnt of dst_cache is correct. * Changed ip_vs_ftp.c to handles established sessions only, which can speed up ftp handling in the LVS/NAT a little bit. * Changed to use records of fixed size and added more boundary checking in all the proc info handlers (of /proc/net/ip_vs, /proc/net/ip_vs_conn, /proc/net/ip_vs_app), so that possible buffer overflow can be avoided. Thanks to Julian for discussions on this issues. * Changed svc->refcnt updates a little bit. Tidy up some stucts in the ip_vs.h and some comments in the code. * Added some statistics functionalities in the ipvs. The statistics information can be accessed by /proc/net/ip_vs_stats. And, thank Julian for a lot discussions on this too. IP Virtual Server Netfilter module - Version 0.2.5 - February 23, 2001 Changes: * Fixed the bug of missing unregistering forward_icmp hook in the module exit, which would make system crash after the ip_vs module is removed and inserted back. * Fixed the wrong alignment of IPVS tcp state transition table by changing to put IP_VS_S_SYNACK after IP_VS_S_LISTEN in the enumeration. Thank Radu-Adrian Feurdean for reporting this bug. * Fixed the bug of forgetting binding ftp data connection with its destination in ip_vs_ftp.c. It would make ftp service of LVS/NAT not working properly. Thank Radu-Adrian Feurdean for the report again. * Fixed to use (char *)th+(th->doff<<2) to get the starting address of DATA in the tcp packet, instead of (char*)&th[1]. Because there may be tcp options in the tcp packet. IP Virtual Server Netfilter module - Version 0.2.4 - February 15, 2001 Changes: * Changed ip_vs_ctl.c to call scheduler's update-service function after a new server is added. * Added the Destination Hashing scheduler. The DH scheduler assigns connections to servers through looking up a statically assigned hash table by their destination IP addresses. * Added the Source Hashing scheduler. The SH scheduler assigns connections to servers through looking up a statically assigned hash table by their source IP addresses. * Alexander Atanasov fixed the lblcr scheduler that when when all servers in the desination set are of weight 0 or are unavailable, falls back to wlc method, instead of return NULL. * IPVSADM upgrade to a new version added --connection -c in command syntax and man page. revisited the SYNOPSIS of ipvsadm command, this one should be more correct, thank Joe and Horms for telling me there is some inconsistency. added the VERSION and SCHEDULERS files. changed "-M [netmask]" to "-M netmask". IP Virtual Server Netfilter module - Version 0.2.3 - January 29, 2001 Changes: * Julian fixed a bug in the LBLCR scheduler by adding the missing (dest->weight>0) condition in the ip_vs_dest_set_max. * Fixed the wrong dep_tristate states in the Config.in of IPVS * Alexander Atanasov has done many testings on the LBLC and LBLCR scheduler, added selecting another server if the assigned dest->weight<=0 in the LBLC scheduler, added the checking on the ip_vs_dest_set_min and ip_vs_dest_set_max return in the LBLCR, otherwise the system would crash. * Changed to when the packet of new connections is marked with non-zero value, lookup the fwmark-based virtual service first, then check normal virtual service if it is not for fwmark-based virtual service. * Added the ip_vs_forward_icmp hook at the NF_IP_FORWARD chain. It is to catch incoming ICMP packets destined for 0.0.0.0/0 but belonging to IPVS connections, and forward them to right servers. IP Virtual Server Netfilter module - Version 0.2.2 - January 26, 2001 Changes: * Julian added the ip_vs_post_routing in the POST_ROUTING hook, which detects outgoing packets for IPVS connections and let them skip the ip_nat_out. Then, VS/NAT works with iptable_nat now. * Added several lines of code in ip_fw_compact.c, which can also identify outgoing packets for IPVS connections and call ip_vs_out to mangle and send them out. Now, VS/NAT works with ipchains too. * Added several lines of instructions about how to compile ipvs code with the kernel in the README. IP Virtual Server Netfilter module - Version 0.2.1 - January 15, 2001 Changes: * Removed some wrong "static" declarations in ip_vs_ctl.c and ip_vs_conn.c. * Ported new stuff of ipvsadm for kernel 2.2 to ipvsadm for kernel 2.4, and upgrade ipvsadm-1.12 for kernel 2.4 to version 1.14. * Added the related Config.in and Makefiles. You can compile ipvs within the kernel after copying those files manually. IP Virtual Server Netfilter module - Version 0.2.0 - January 10, 2001 Changes: * Added a "packet_xmit" function in the ip_vs_conn entry, and remove the original ip_vs_forward, which can save several CPU cycles in the packet forwarding and is more flexible. * Added the "cache_bypass" feature. when no cache server is available, cache_bypass sysctl is up and iph->daddr is RTN_UNICAST (not local), forward packets to the original destination directly. * Clarified all the debug levels in the code. * Tidy up the code, such as adding new functions, changing the function names, updating the comments. IP Virtual Server Netfilter module - Version 0.1.2 - December 19, 2000 Changes: * Julian added using destination cache to speed up route lookup for outgoing packets. The feature can be controlled by the following sysctl: /proc/sys/net/ipv4/vs/use_dst It is enabled by the default. * The bug of repeated sysctl ID for NET_IPV4_VS_LBLC_EXPIRE and NET_IPV4_VS_LBLCR_EXPIRE was fixed. IP Virtual Server Netfilter module - Version 0.1.1 - November 30, 2000 Changes: * Added the Locality-Based Least Connection with Replication scheduler. In the LBLC scheduling, the load balancer maintains mappings from a target to a cache node, when the cache node is overloaded, the target may be remapped to the current least loaded node. The target will cause the node overloaded again, then will be remapped to other node, and so on. This is the "hot spot" problem, it may lead to a lot of replications in most nodes. So, the LBLC with Replication algorithm is introduced to solve this problem, it maintains mappings from a target to a set of nodes that can solve this target. So, the "hot spot" can be solved by a set of nodes, then "hot spot" can seldom overload one and another among all the nodes. IP Virtual Server Netfilter module - Version 0.1.0 - November 26, 2000 Changes: * Julian added ECN support in IPVS tunnelling transmitter * Ported the Locality-Based Least Connection scheduler from IPVS code for kernel 2.2. Julian replaced del_timer function call with with del_timer_sync to avoid the possible race between timer handler and del_timer thread in SMP. * Julian and Wensong fixed the possible oops in the todrop_entry(). When the incoming packet counter of a very busy connection exceeds 2^32, accessing an array with a negative index leads to oops. IP Virtual Server Netfilter module - Version 0.0.5 - September 6, 2000 Changes: * Rusty fixed the checksum bug in ip_vs_skb_replace of ip_vs_app.c iph->tot_len should be updated before the checksum is calculated. * Rusty made all local functions and variables static, and cleaned up the modules without #ifdef's. * Julian added the real service checking back in ip_vs_out. * Removed the ip_vs_register_scheduler_module() and the ip_vs_unregister_scheduler_module() in the ip_vs_sched.c * Julian and Wensong explored race of ipvs connection creation/ deletion. The original wrong locking was fixed in the ip_vs_random_dropentry(). Changed to increase the reference counter of a new connection entry before hashing it, so that while one CPU is creating a new entry, the other CPU may run the ip_vs_random_dropentry() but cannot drop it. * Julian fixed the bug in the WRR scheduler, server with weight zero is selected when all server weights (>= 2 servers) are zero. Thank Ratz for reporting the bug. * Added the "-s tcp tcpfin udp" option in the ipvsadm.c, so that the timeout values for IPVS connections (tcp tcpfin udp) can be changed in the kernel. * Fixed the compiling error in sysctl_ip_vs_debug_level when the CONFIG_IP_VS_DEBUG is turned off. IP Virtual Server Netfilter module - Version 0.0.4 - July 30, 2000 Changes: * Rusty tidy up the ipvs code during the Ottawa Linux Symposium 1) Fix `large ftp' bug: more precision in sequence delta. 2) Reorder struct ip_vs_conn to save some space. 3) Firewall marks must be `unsigned long' not __u32. 4) kfree_s changed to kfree * Julian changed kmalloc connection allocation to slab cache allocation. It's good for performance. * Added the decreasement of the ip_vs_conn_no_cport_cnt if the flag is set with IP_VS_CONN_F_NO_CPORT. It can avoid the wrong counting when IP_VS_CONN_F_NO_CPORT connection entries are expired or dropped. IP Virtual Server Netfilter module - Version 0.0.3 - July 6, 2000 Changes: * Made IPVS application module and ip_vs_ftp module work. * Changed ip_vs_in hook from NF_IP_PRE_ROUTING to NF_IP_LOCAL_IN. Then, there is no need to call ip_defrag(), and no source routing problem. Firewalling and policy routing rules apply to IPVS. The structure looks better. * Splitted some code of ip_vs_in into ip_vs_nat_xmit and added MTU checking there. It may exist that application modules may extend the sk_buff into a new one and the length of the new one is larger than MTU, but this problem should rarely happen. * Changed the checksum checking is only done for VS/NAT just before mangling pakcet. VS/TUN and VS/DR don't modify packets, so let real servers do checksum checking. It can speed up packet handling in VS/TUN and VS/DR. * Fixed many small bugs, and changed lots of debugging display nicer. The functionality of IPVS for kernel 2.4 is almost implemented. The rest things to do are bug fixes, performance tuning, and making it work with other netfilter modules harmonly. Currently, if ip_conntrack module is inserted, then there will be two tracking for each connection. The iptable_nat module will make the passive FTP of VS/NAT not working correctly. The above changes were done with Julian together. Thank Julian for good suggestions and many bug fixes. Note: The version 0.0.2 is for internal developemnt, so it is not released publicly. IP Virtual Server Netfilter module - Version 0.0.1 - May 31, 2000 Changes: * Port the IPVS patch for kernel 2.2 as a Netfilter module for kernel 2.4. Lots of code were rewritten in order to adapt it to the Netfilter framework, and lots of code were taken from the ip masquerading code of kernel 2.2. Currently, this draft version can just work basically. There are still lots of work to do, such as more rewrite, bug fixes and performance improvements for kernel 2.4. * Julian adapted locking for kernel 2.4 * Julian adapted slow timer handling for kernel 2.4 * Julian made the module reference correct ChangeLog of Virtual Server patch for Linux 2.2 =============================================== Virtual Server patch for Linux 2.2.15 - Version 0.9.13 - May 25, 2000 Changes: * Changed that the key of real server hash table is generated from , instead of , and changed the check in __ip_vs_lookup_real_service(). It should work correctly for fwmark service. * Added the missing "INIT_LIST_HEAD(&dest->d_list);" in ip_vs_rs_unhash(). * Changed that ip_vs_wrr_init_svc() returns -ENOMEM instead of ENOMEM if no memory is available, just in order to keep good format of return value. * Added the ftp service and port zero service counters, which can speed up service lookup. * Restrict real server ports covered from the ftp service. The port number of ftp data connections on real servers can only be 20 or >= PROT_SOCK(1024). * Split some duplicate code into __ip_vs_del_service() The above changes were made by Julian and Wensong. Virtual Server patch for Linux 2.2.14 - Version 0.9.12 - May 2, 2000 Changes: * The condition matching order in __ip_vs_in_get & __ip_vs_out_get was optimized, it should speed up the entry lookup when the row is long. Thank Julian! * Fixed the silly bug that I forgot adding an sysctl ID for the /proc/sys/net/ipv4/vs/am_droprate. * Julian fixed the wrong return (EINVAL) to the correct one (-EINVAL), when the option length is less than the sizeof the ip_masq_ctl entry. Virtual Server patch for Linux 2.2.14 - Version 0.9.11 - April 17, 2000 Changes: * Fixed icmp handling bug introduced in adding the fwmark feature * Made the dest lookup correctly in trash for both normal service and fwmark service. * Changed that the ICMP_DEST_UNREACH packet is sent to clients only if its entry doesn't exist and it is not RST or not of TCP, the ICMP_DEST_UNREACH packet is sent to real servers only if its entry doesn't exist and it is not RST or not of TCP. * Horms tidy up some description and grammar of ipvsadm man page. The above changes were made by Julian, Horms and Wensong together Virtual Server patch for Linux 2.2.14 - Version 0.9.10 - April 9, 2000 Changes: * Julian added the droprate and secure_tcp defense strategies. * The dropentry defense strategy was revisited. * The fwmark service lookup was added by Horms, Julian and Wensong Use a firewall-marking to denote a virtual service instead of a triplet . The marking of packets with a firewall-mark is done by firewalling code. This feature can be used to build a virtual service assoicated to different IP addresses or port numbers, but sharing the same real servers, such as multiple-homed LVS. Virtual Server patch for Linux 2.2.14 - Version 0.9.9 - March 16, 2000 Changes: * Fixed the OUTPUT state transition table It entered from RS to ES while receiving SYN packet, which is not right and will cause that randrom drop won't be effective for VS/NAT. * Added random drop of UDP entries too before out of memory Thank Julian for his cute comments. Virtual Server patch for Linux 2.2.14 - Version 0.9.8 - March 13, 2000 Changes: * Added random drop of syn entries before running out of memory When available memory is less than 1024 pages, randomly scan 1/16 of table to drop entries that are in SYN_RECV state. To activate this random drop functionality, echo 1 > /proc/sys/net/ipv4/ip_vs_randomdrop it will be hard for distributed syn flooding attack tools to make the LVS box run out of memory. * Fixed a negative counter bug because of wrong invalid template The invalid template like is changed to . Virtual Server patch for Linux 2.2.14 - Version 0.9.7 - January 19, 2000 Changes: * Just resolve a patch rejection on Configure.help for 2.2.14 Virtual Server patch for Linux 2.2 - Version 0.9.7 - December 22, 1999 Changes: * Fixed the huge timeout entry bug when destinations are unavailable When the destination server of a packet is found unavailable, the packet is droped silently but the entry is forgotten to be added back to the slow timer table. It would generate the entries of huge timeout. Thank Julian for the bug. * Changed two IP_VS_ERR calls to IP_VS_DBG Since the ipvsadm would report the error information when deleting a nonexist destionation or adding an existing service, there is no need to report error message in kernel. Thank Julian again for the change. * Added the sysctl_ip_always_defrag counting in ip_masq_new_vs This is for the coming kernel patch 2.2.14, where the wrong sysctl_ip_always_defrag handling is fixed. Virtual Server patch for Linux 2.2 - Version 0.9.6 - December 7, 1999 Changes: * Invalidate a persistent template when its dest is unavailable We define templates like (persistence for a single service) or (persistence for all services) are valid, and templates like are invalid. When new connection arrives and the destination of its template is not available, invalidate the template, then create a new template with new destination, and new connection is served. * Fixed the wrong debugging information in ip_vs_forward Virtual Server patch for Linux 2.2 - Version 0.9.5 - November 28, 1999 Changes: * Fixed the undefined variable bug in the IP_VS_DBG Due to my carelessness, an undefined variable was left in the IP_VS_DBG statement of the ip_vs_dr_xmit function. Thank Roberto Nibali for reporting. * Changed ICMP_PROT_UNREACH to ICMP_PORT_UNREACH in ip_vs_leave When virtual service is available but no destination is available, The ICMP_PORT_UNREACH icmp packet is sent to notify the client that the service is not available. Since IPVS is in IP layer, the TCP socket has been created, the TCP RST packet cannot be sent for TCP services, instead that ICMP_PORT_UNREACH is sent, no matter it talks TCP/UDP. Thank Julian. * Added port zero support for persistent services For some applications, there are more than one service, once a client is assigned to a real server for the first service, requests for other services from the same clients must be sent to the same server. Port zero is added for this kind of persistent services. * Fixed the bug that virtual ftp service blocks other services When virtual ftp service is presented and packets destined for other services not listed in ipvs table arrives, wrong masq entries will be created and those services are blocked. * Fixed the (null) print for unknown services in ipvsadm Thank Julian for reporting. Virtual Server patch for Linux 2.2 - Version 0.9.4 - November 10, 1999 Changes: * Julian fixed the fatal return bug of ip_vs_leave() Since some code of last version ipvs is changed, ip_vs_leave should return -2 instead of -3 if no virtual service is found. * Added the IPSKB_REDIRECTED flag The skb is set with the IPSKB_REDIRECTED and IPSKB_MASQUERADED flag, so that the system can detect infinite loop of TUNNELED/ DROUTED packets in the ip_local_deliver caused by misconfiguration. For example, user might configure the following: ipvsadm -a -t VIP:http -r -i ifconfig up then packets for VIP:http is tunneled to its own interface, which will causes infinite loop. * Fixed the bug that freed skb may be used to masq_set_state In the original ip_fw_demasquerade function, masq_set_state was called after ip_vs_forward, and ip_vs_forward may free the skb, so masq_set_state may operate the already freed skb. The current solution is just to simply do masq_set_state before ip_vs_forward. No matter whether the packet is forwarded successfully or not, the masq state will be updated. Although it brokes the original sematics, it won't lead to serious errors. We look forward to fixing it under the Rusty's netfilter framework both for correctness and modularization. :-) Many thanks must go to Julian for his very cute comments to the ipvs 0.9.3 code. He also raised a question, could we simply use ip_route_output to skip IPv4 forwarding and firewall to tunnel/ droute packets for a little bit performance, or should we be back to ip_route_input for correctness? I am still thinking about it. Virtual Server patch for Linux 2.2 - Version 0.9.3 - November 7, 1999 Changes: * Adapted the patch for kernel 2.2.13 Since the ntohl and like were changed to unsigned int(because the unsigned long int is 64-bit these days), some code in VS patch is modified for this change, and the compiling warnings and unnecessary casting can be avoided. * Changed the masq timeout type and the maximum persistent timeout The type of masq timeout was changed from 'unsigned' to 'unsigned long', in order to keep it the same as the type of timer_struct expires, then masq timeout will be 64-bit on 64-bit platforms. The maximum persistent timeout was changed from one year to one month, because this is enough. Thank Julian for the suggestions. * Added ICMP handling for IPVS The incoming ICMP packets for virtual services will be forwarded to the right real servers, and outgoing ICMP packets from virtual services will be altered and send out correctly. This is important for error and control notification between clients and servers, such as the MTU discovery. Sorry for adding this stuff so late, because I used to stupidly think that it is not easy to add ICMP handling for IPVS. After spending a couple of hours reading the textbooks and the masq code, I found that it was quite easy to add this stuff. Sorry! * Changed the tunnel/dr/local forwarding without doing masq_skb_cow Some orders in the ip_fw_demasquerade and ip_fw_demasq_icmp functions, so that the masq skbuff copy-on-write can be avoided in the tunnel/ dr/local forwarding methods. This improves performance for the tunnel/dr/local forwarding methods. * Use vmalloc to allocate big hash table. The big IPVS hash table of 256K entries or more can be allocated now. Virtual Server patch for Linux 2.2 - Version 0.9.2 - October 17, 1999 Changes: * Added support for netmasks with persistence The client source address is masked with this netmask for the purpose of accessing the templates. Added a new port to the service structure and changed ipvsadm to support this. Defaults to a 255.255.255.255, which emulates the old behaviour. (Lars Marowsky-Bree ) * Fixed the bug that server status checking doesn't work for LVS/NAT, and changed some comestics things for debugging. Thank Julian for the fix. Virtual Server patch for Linux 2.2 - Version 0.9.1 - October 6, 1999 Changes: * Fixed the counting bug in ip_vs_unbind_masq again Don't touch counters for templates. * Removed extra read_unlock in __ip_vs_lookup_service * Changed not to restart template timers if dest is unavailable If the client actively send packets when the destination is unavailable, the masq template can expire. * Added the destination trash The destination trash is used to hold the destinations that are removed from the service table but are still referenced by some masq entries. The reason to add the destination trash is when the dest is temporary down (either by administrator or by monitor program), the dest can be picked back from the trash, the remaining connections to the dest can continue, and the counting information of the dest is also useful for scheduling. * Added the ip_vs_leave function It is called by ip_fw_demasquerade when the matched service is avaiable but no destination is available for a new connection, to drop the packet. This should be a good behavior. * Changed drasticly removing the masq to silently dropping packets and keeping the masq in expire, when its destination is not available. It is a good behavior, when the destination is temporary down. The above fixes and changes won't be possible without Julian Anastasov's fixes and suggestions. Thank Julian! * Added the handling of weight=0 in every scheduler The destination with weight=0 is "quiesced" and will not receive any new connection, but will still serve the existing connections. This feature is useful to cool down the overloaded servers or to get some servers out of service for maintenance. * Added the update_service function in every scheduler When the destination list of a service is modified, the update_service function is called to reset the scheduling pointer, so that the scheduling pointer won't point to the freed destination. * Changed some IP_VS_ERR to IP_VS_DBG in the ip_vs_tunnel_xmit * Added different timeout support for persistent service Users can specify different timeout values for their different persistent services. * Fixed the bug that persistent service cannot be edited * Changed the output of ip_vs_procinfo for the new version of ipvsadm. Virtual Server patch for Linux 2.2 - Version 0.9.0 - September 24, 1999 Changes: * Added the hash table for virtual services It will greatly speedup the lookup of services. * Added new persistent service handling The template is looked up only if the service that the packet is destined to is persistent, so it is more efficient. For all the persistent services except FTP, we create a masq template like . So, the persistent services won't disturb each other, and it fixes the wrong accounting bug for different persistent services. FTP is a very complicated network protocol, and it uses control connection and data connections. For active FTP, FTP server initilizes data connection to the client, its source port is often 20. For passive FTP, FTP server tells the clients the port that it passively listens to, and the client issues the data connection. In the tunneling or direct routing mode, the load balancer is on the client-to-server half of connection, the port number is unknown to the load balancer. So, a template masq like is created for persistent FTP service. * Changed the destination lists to the d-linked lists * Changed the scheduler list to the d-linked list * Added back the least connection scheduling module. Virtual Server patch for Linux 2.2 - Version 0.8.3 - September 8, 1999 Changes: * Fixed the missing unlock bug in ip_vs_schedule. If no virtual service is found in ip_vs_schedule, this missing unlock bug will make system crash. * Fixed the uncounting bug in creating masqs by template. Missing to counter connections when creating masqs by template. * Don't touch counters in ip_vs_unbind_masq for templates Thanks must go to Julian Anastasov for the three fixes above. * Changed some condition orders for a bit performance * Changed some cosmetic things for debugging Virtual Server patch for Linux 2.2 - Version 0.8.2 - September 5, 1999 Changes: * Fixed the the IP_MASQ_F_VS_INACTIVE cleared bug after editing dest. Thank Julian Anastasov for the fix. * Added the separate inactive connection counter for each dest The WLC sheduler can use this counter directly for scheduling. And, the masq template won't be counted in inactive connections. Thank Julian Anastasov for the suggestion. * Changed all the schedulers modules to return server dest directly, and ip_vs_schedule creates new masq entry itself. Virtual Server patch for Linux 2.2 - Version 0.8.1 - September 2, 1999 Changes: * Uncomment a few statement to make virtual FTP via NAT really work. Virtual FTP service via NAT really work well no matter it is in active or passive mode. But, remember to "insmod ip_masq_ftp" before using FTP service through VS-NAT. * Remove some commented out block. The code looks nice. :) Virtual Server patch for Linux 2.2 - Version 0.8 - September 1, 1999 Changes: * Added the persistent port feature. Users can specify whether the virtual service port is persistent or not. It is more flexible. The original PCC scheduling is removed. * Added the dest server status checking. The server status is checked before forwording a packet. If the server is not available(down or put out of service), the packet will be dropped and the client will be notified immediately. The server status is also checked while generating a masq entry based on the masq template. If not available, the new entry won't be created. * Added some code in ip_masq_ftp.c to handle virtual FTP service for VS-NAT. The passive handling code in ip_masq_ftp.c never works. * Fixed stepping to mSR after SYN in INPUT_ONLY table. Thank Julian Anastasov for doing it. It make much much harder that a LinuxDirector is synflooded to run out of memory. * Fixed huge masq expire bug for after bad checksum. Thank Julian Anastasov for fixing it. * Added the IP_MASQ_F_VS_INACTIVE flag and fixed the connection counter Thank Julian Anastasov for the suggestion and fix example. * Fixed the incorrect lookup in hash table. The ms=NULL statement was forgot if no entry is found, this makes the incorrect lookup, which may lead to huge masq expire. Stupid mistake, but the result is serious. * Fixed the incorrect slow timer vector layout Correct layout and more efficient to use memory. * Fixed the bug of slow timer being added twice for masq template Virtual Server patch for Linux 2.2 - Version 0.7 - July 9, 1999 Changes: * Added a separate masq hash table for IPVS. * Added slow timers to expire masq entries. Slow timers are checked in one second by default. Most overhead of cascading timers is avoided. With this new hash table and slow timers, the system can hold huge number of masq entries, but make sure that you have enough free memory. One masq entry costs 128 bytes memory effectively (Thank Alan Cox), if your box holds 1 million masq entries (it means that your box can receive 2000 connections per second if masq expire time is 500 seconds in average.), make sure that you have 128M free memory. And, thank Alan for suggesting the early random drop algorithm for masq entries that prevents the system from running out of memory, I will design and implement this feature in the near future. * Fixed the unlocking bug in the ip_vs_del_dest(). Thank Ted Pavlic for reporting it. Virtual Server patch for Linux 2.2 - Version 0.6 - July 1, 1999 Changes: * Fixed the overflow bug in the ip_vs_procinfo(). Thank Ted Pavlic for reporting it. * Added the functionality to change weight and forwarding (dispatching) method of existing real server. This is useful for load-informed scheduling. * Added the functionality to change scheduler of virtual service on the fly. * Reorganized some code and changed names of some functions. This make the code more readable. Virtual Server patch for Linux 2.2 - Version 0.5 - June 22, 1999 Changes: * Fixed the bug that LocalNode doesn't work in vs-0.4-2.2.9. Thank Changwon Kim for reporting the bug and pointing me the checksum update problem in the code. * some code of VS in the ip_fw_demasquerade was reorganized so that the packets for VS-Tunneling, VS-DRouting and LocalNode skip the checksum update. This make the code right and efficient Virtual Server patch for Linux 2.2 - Version 0.4 - June 1, 1999 Most of the code was rewritten. The locking and refcnt was changed The violation of "no floats in kernel mode" rule in the weighted least-connection scheduling was fixed. This patch is more efficient, and should be more stable. Virtual Server patch for Linux 2.2 - Version 0.1~0.3 - May 1999 Peter Kese ported the VS patch to kernel 2.2, rewrote the code and loadable scheduling modules. ChangeLog of Virtual Server patch for Linux 2.0 =============================================== Virtual Server Patch for Linux - Version 0.9 - May 1, 1999 Differences with virtual server patch version 0.8: * Add Virtual Server via Direct Routing This approach was first implemented in IBM's NetDispatcher. All real servers have their loopback alias interface configured with the virtual IP address, the load balancer and the real servers must have one of their interfaces physically linked by a HUB/Switch. When the packets destined for the virtual IP address arrives, the load balnacer directly route them to the real servers, the real servers processing the requests and return the reply packets directly to the clients. Compared to the virtual server via IP tunneling approach, this approach doesn't have tunneling overhead(In fact, this overhead is minimal in most situations), but requires that one of the load balancer's interfaces and the real servers' interfaces must be in physical segment. * Add more satistics information The active connection counter and the total connection counter of each real server were added for all the scheduling algorithms. * Add resetting(zeroing) counters The total connection counters of all real servers can be reset to zero. * Change some statements in the masq_expire function and the ip_fw_demasquerade function, so that ip_masq_free_ports won't become abnormal number after the masquerading entries for virtual server are released. * Fix the bug of "double unlock on device queue" Remove the unnecessary function call of skb_device_unlock(skb) in the ip_pfvs_encapsule function, which sometimes cause "kernel: double unlock on device queue" waring in the virtual server via tunneling. * Many functions of virtual server patch was splitted into the linux/net/ipv4/ip_masq_pfvs.c. * Upgrade ippfvsadm 1.0.2 to ippfvsadm 1.0.3 Zeroing counters is supported in the new version. The ippfvsadm 1.0.3 can be used for all kernel with different virtual server options without rebuilding the program. Virtual Server Patch for Linux - Version 0.8 - March 6, 1999 Differences with virtual server patch version 0.7: * Add virtual FTP server support The original ippfvs via IP tunneling could not be used to build a virtual FTP server, because the real servers could not establish data connections to clients. The code was added to parse the port number in the ftp control data and create the corresponding masquerading entry for the coming data connection. Although the original ippfvs via NAT could be used to build a virtual server, the data connection was established in this way. Real Server port:20 ----> ippfvs: allocate a free masq port -----> the client port It is not elegent but time-consuming. Now it was changed as follows: Real Server port:20 ----> ippfvs port: 20 ----> the client port * Change the port checking order in the ip_fw_demasquerade() If the size of masquerade hash table is well chosen, checking a masquerading entry in the hash table will just require one hit. It is much efficient than checking port for virtual services, and there are at least 3 incoming packets for each connection, which require port checking. So, it is efficient to check the masquerading hash table first and then check port for virtual services. * Remove a useless statement in the ip_masq_new_pfvs() The useless statement in the ip_masq_new_pfvs function is ip_masq_free_ports[masq_proto_num(proto)]++; which may disturb system. * Change the header printing of the ip_pfvs_procinfo() Virtual Server Patch for Linux - Version 0.7 - Febuary 10, 1999 Differences with virtual server patch version 0.6: * Fix a bug in detect the finish of connection for tunneling or NATing to the local node. Since the server reply the client directly in tunneling or NATing to the local node, the load balancer (LinuxDirector) can only detect a FIN segment. It is mistake that the masq entry is removed only if both-side FIN segments are detected, and then the masq entry expires in 15 minutes. For the situation above, the code was changed to set the masq entry expire in TCP_FIN_TIMEOUT (2min) when an incoming FIN segment is detecting. * Add the patch version printing in the ip_pfvs_procinfo() It would be easy for users and hackers to know which virtual server patch version they are running. Thank Peter Kese for the suggestion. Virtual Server Patch for Linux - Version 0.6 - Febuary 2, 1999 Differences with virtual server patch version 0.5: * Add the local node feature in virtual server. If the local node feature is enabled, the load balancer can not only redirect the packets of the specified port to the other servers (remote nodes) to process it, but also can process the packets locally (local node). Which node is chosen depends on the scheduling algorithms. This local node feature can be used to build a virtual server of a few nodes, for example, 2, 3 or more sites, in which it is a resource waste if the load balancer is only used to redirect packets. It is wise to direct some packets to the local node to process. This feature can also be used to build distributed identical servers, in which one is too busy to handle requests locally, then it can seamlessly forward requests to other servers to process them. This feature can be applied to both virtual server via NAT and virtual server via IP tunneling. Thank Peter Kese for idea of "Two node Virtual Server" and his single line patch for virtual server via IP tunneling. * Remove a useless function call ip_send_check in the virtual server via IP tunneling code. Virtual Server Patch for Linux - Version 0.5 - November 25, 1998 Differences with virtual server patch version 0.4: * Add the feature of virtual server via IP tunneling. If the ippfvs is enabled using IP tunneling, the load balancer chooses a real server from a cluster based on a scheduling algorithm, encapsules the packet and forwards it to the chosen server. All real servers are configured with "ifconfig tunl0 up". When the chosen server receives the encapsuled packet, it decapsules the packet, processes the request and returns the reply packets directly to the client without passing the load balancer. This can greatly increase the scalability of virtual server. * Fix a bug in the ip_portfw_del() for the weighted RR scheduling. The bug in version 0.4 is when the weighted round-robin scheduling is used, deleting the last rule for a virtual server will report "setsockopt failed: Invalid argument" warning, in fact the last rule is deleted but the gen_scheduling_seq() works on a null list and causes that warning. * Add and modify some description for virtual server options in the Linux kernel configuration help texts. Virtual Server Patch for Linux - Version 0.4 - November 12, 1998 Differences with virtual server patch version 0.3: * Fix a memory access error bug. The set_serverpointer_null() function is added to scan all the existing ip masquerading records for its server pointer which points to the server specified and set it null. It is useful when administrators delete a real server or all real servers, those pointers pointing to the server must be set null. Otherwise, decreasing the connection counter of the server may cause memory access error when the connection terminates or timeout. Virtual Server Patch for Linux - Version 0.3 - November 10, 1998 Differences with virtual server patch version 0.2: * Change the simple round-robin scheduling to the weighted round-robin scheduling. Simple is a special instance of the weighted round-robin scheduling when the weights of the servers are the same. * The scheduling algorithm, originally called the weighted round-robin scheduling in version 0.2, actually is the weighted least-connection scheduling. So the concept is clarified here. * Add the least-connection scheduling algorithm. Although it is a special instance of the weighted least-connection scheduling algorithm, it is used to avoid dividing the weight in looking up servers when the weights of the servers are the same, so the overhead of scheduling can be minimized in this case. * Change the type of the server load variables, curr_load and least_load, from integer to float in the weighted least-connection scheduling. It can make a better load-balancing when the weights specified are high. * Merge the original two patches into one. Users have to specify which scheduling algorithm is used, the weighted round-robin scheduling, the least-connection scheduling, or the weighted least-connection scheduling, before rebuild the kernel. * Change the ip_pfvs_proc function to make the output of the port forwarding & virtual server table more beautiful. Virtual Server Patch for Linux - Version 0.2 - May 28, 1998 Differences with virtual server patch version 0.1: * Add the weighted round-robin scheduling patch. Virtual Server Patch for Linux - Version 0.1 - May 26, 1998 * Implement the infrastructure of virtual server. * Implement the simple round-robin scheduling algorithm.