What is an Electronic Document Management System and why is it required?

A document is any readable material that is used for business processes. Documents can be in multiple formats – printed pages, word processing documents, spread sheet, fax, presentation documents, photos, internet pages, medical images, AutoCAD drawings etc. We are handling more of electronic documents these days and then converting it to paper based documents, where required, by printing them. What if the entire process can be handled electronically without having to use papers at all? Then comes the storage and retrieval of such printed documents – it always happens that the most important file we are looking for is always not available (like a copy of a purchase order/ invoice etc). So, if all the important documents are stored electronically, it would be much easier to locate an important document that we may want to reference at that instant. And then comes the electronic documents (like word documents, power point files, auto cad files, emails etc) – If the critical electronic documents are also indexed by a central document management system, they could also be searched using some keywords in their contents, tags like date of creation, person who saved etc.

So, basically an electronic document management system refers to the creation, capture, organization, storage, retrieval, manipulation and controlled circulation of documents in electronic format.

The advantages of electronic document management systems are numerous. Some of them are given below:

¤ The documents stored in EDMS can be easily searched and viewed from any computers ensuring their instant and fast access/availability.
¤ As the printed documents are scanned, they would retain their original look, colour etc.
¤ Optical Character Recognition technology is useful to digitize the entire text of a document and allows the users to search using any textual term contained in them.
¤ Documents can be quickly shared for collaboration on the network.
¤ Electronic documents can be stored in multiple places (useful for disaster recovery).
¤ The real estate costs of storing paper based documents is reduced.
¤ Internal processes like approvals etc. can be done faster as most of the companies are multi-location based and it would take a long time to send and get back individual documents.

Creation/Capture/Organization:

A document management system consists of an imaging server which contains the EDMS software that allows to store, retrieve and access documents based on any combination of indexes, text (if the scanned documents have gone through an Optical Character Recognition process). EDMS generally use database management systems like MySQL, Oracle etc to store and organize the documents. Some EDMS can be integrated with the existing database system used by the company.

People are also an integral part of the Document Management System as they have the knowledge about the type of documents, how to segregate them in to various groups, how the communication lines flow and take decisions as to which processes/work-flows need to be automated. Equally important is the business processes which define the organization and the flow of documented information between people within and outside the organization.

Printed documents are generally scanned by using specialized high volume scanners or certain Multi-Function Printers/Copier’s etc which can be integrated with the document management systems. These documents are indexed using certain parameters like date of creation, person who created, time of creation etc, and they can be organized in to related groups of folders.

Some EDMS allow automated capture and storage of incoming/ outgoing email messages and their attachments in to the document management system folders so that they can be stored and accessed whenever required.

Storage and Retrieval:

The electronic documents are generally stored in servers and other permanent storage. The current documents (that are used frequently or recently at that point of time) can be stored in a server/ hard disks and the archived documents can be kept on tape drives, CD’s, DVD’s, Storage Area Networks, CD/Optical Jukeboxes etc. Mostly the documents are stored in pdf format (scanned ones) so that they can be instantly accessed, viewed and searched from any PC/workstation.

Generally, some security features are applied to the accessing of the documents to control which users can access which documents at what times. And only one user is allowed to modify a given document at a point of time (as the documents can be accessed by multiple users at the same time).

EDMS allow locating/searching documents based on the text they contain (if the scanned pages have gone through an OCR) or through any of the indexed parameters. Some times, compression mechanisms are applied to images before storing them in order to reduce the disk space required for storing them.

Some EDMS softwares allow the search module can be integrated in to the existing software applications that the employees use on a daily basis like CRM/ERP modules etc.

Manipulation and Controlled Circulation:

The EDMS check if the stored documents have undergone any changes and intimate to the respective personnel concerned when some changes happen to important documents before saving them. Some times, approvals might be needed to be obtained before saving the changes to the saved documents, and EDMS allow to do the same.

One of the biggest advantages of EDMS is to quickly share the documents over the network (Internet/Intranet). Another important function of EDMS is to apply work-flow routing to the documents. All companies have some processes where important tasks are required to go through multiple approvals. An Electronic Document Management System automates this process and lets the  personnel to give approvals over the network. You could imagine how much time a manual system of approvals would have consumed with printed paper and multi-location offices.

Some EDMS can also be integrated with the web and the documents can be presented to the user via a normal browser and displayed as dynamically generated HTML pages.

excITingIP.com

In case you have any questions, you can contact us using the contact form or leave a comment below. You can also subscribe with your email address (on the right side of this site) to get intimated when a new article is published on this site.

What are Application Load Balancers?

Application load balancers are hardware devices that sit in between a bunch of servers in the data centre and the outside world (people requesting access to the servers). They balance the load (traffic, sessions) across a bunch of servers (applications actually) and make those multiple servers (and applications) look like one big server to the outside world. Actually the applications are replicated across multiple servers and the incoming requests are routed to any of them, based on multiple parameters. This is required for multiple reasons, chief of them given below.

Why are Application Load Balancers required?

Scalability: Suppose a single server is used to host a website or has some application that is frequently used by large number of users. There would come a point when the servers maximum capacity (concurrent connections, processor speed, bandwidth limit etc) is reached and multiple servers are required to be used. So, an application load balancer decides which user goes to which server. In other words, it distributes and balances the load (incoming process requests) across multiple servers and more importantly allows the addition of multiple servers.

High Availability: Suppose you are running a very critical application (like an ecommerce website) on a single server, there is always a possibility that there could be an application failure or server hardware failure. In those cases, it is better to run the same application in multiple servers – both for load balancing purposes and to avoid complete disruption of services in case a server/application fails as the load balancer can automatically identify if a server/application is down and not route any connection requests to it, until it is up and running once again.

Control: Load balancers not only determine if a server is available or not, but they also predict the approximate usage levels of each server/ application. This is required to decide where to forward the incoming requests – it is better to forward them to the server that is being least used at that point of time, for example.

A little History about Application Load Balancers:

Before we go in to the features of application load balancers, we would see a little history to understand the current implementations better.

Initially methods like DNS Round Robin was used to distribute the load across servers. It was a simple method where, if three servers are present, the connections would be sent first to one server, then the second and then third – one after another and the next time the order could differ. While this method was good at distributing load across servers, it was not actually load balancing. There was also no way of determining if a server was down so Availability was not always 100% as it depended on manual methods to determine that. There was another problem: Clients tend to cache the server information (including IP address) and go back to the same server they used before.

Then load balancing was built into the application software. Here, all the client requests go to the cluster IP first and then it is distributed to one of the available and the most suitable physical IP address (of the server/ application port). The problem of High Availability and load balancing is solved as the application developers would know the health of an application (if it is down) and can determine the connection density based on real time parameters to apply load balancing algorithms that is best for the particular application. But the problem is the fact that load balancing is entirely dependant on the application vendors (which might not be provided at all by some of them) and it needs to be done separately for each application. It becomes complex in a virtual server environment.

Network based Application Load Balancing Hardware:

The following steps are performed by network based application load balancing hardware devices which sit in between the leased lines (users) and the host servers to do load balancing of applications.

¤ When the user attempts to connect to the servers, the load balancer accepts the connections on behalf of the server (through a virtual IP address), changes the destination IP address to the physical server IP address and port numbers and forwards the request to the appropriate server.
¤ The server accepts the request, processes it and replies back to the load balancer.
¤ The load balancer now forwards this reply after changing the virtual IP address to the actual user IP address in the destination field so that the user thinks that the reply has come directly from the server.

Application level load balancing: A load balancer can make a distinction between a physical server and the application services running on it. It individually interacts with the applications instead of the underlying hardware, giving the load balancer the ability to load balance at application level instead of server level. Load balancers can balance the load of multiple applications, uniformly.

Health Monitoring for HA: Load balancers can individually verify if a server is working or not. They do this by conducting multiple tests (with increasing complexity) on the servers like Pinging etc. Generally this is done regularly and before the packets are sent to the server in order to ensure HA.

Load balancing parameters: The decision to route a connection request to a particular server over the other servers is taken based on a lot of real time parameters that are measured by application load balancers like load, response times, usage and utilization statistics, current connection counts, host utilization monitors, and a lot more based on the vendor. They also enable dynamic load balancing – sending more traffic to bigger servers (having more processing power) than smaller servers.

Connection persistence: After deciding to connect a particular user to a particular server , the load balancer still ought to determine if the traffic that follows afterwards from that user needs to be load balanced or not. It the session is a longer TCP connection (like FTP), then it should not get load balanced. If the session consists of multiple short lived TCP connections (like http) then it could be load balanced. But for certain http sessions like e-commerce applications, it is important that the users need to be connected to the same server. In such cases, first the applications need to be identified, and the decision to load balance or not, could be taken based on parameters like user name etc (instead of IP address as the proxy servers and NAT gives same IP address for multiple users) which are more permanent and can be read from the incoming packets.

excITingIP.com

In case you have any questions, you could get in touch with us using the contact form or leave a comment below. You could also participate in the discussions in the Forum.

There may be different type of packets traversing through the WAN links of an organization. As data centres are consolidated to a central location, the WAN links almost become an extension of LAN and as a network is only as fast as its slowest link, it becomes more and more important to optimize the performance of a WAN link. One of the main parameters to optimize in a WAN link is the bandwidth. Whatever is the available bandwidth, there are always some chatty non-critical applications that choke WAN links. There are always some critical/real-time applications that suffer because of the bandwidth occupied by the chatty applications. So, there is a need to identify such chatty/non-critical applications and optimize their performance over the WAN links that connect an organization and its data centres. The following are some of the steps followed by bandwidth management/optimization vendors to achieve that.

Bandwidth Categorization:

Traffic can be categorized based on a lot of criterion. The most popular methodology is to categorize traffic based on application. But they can also be categorized based on other factors like port number, protocol, URL, MPLS labels, IP or MAC address, subnet, VLAN, database type, source, destination etc. Eg. ERP traffic to and from a specific server, traffic with an MPLS label of 8, traffic from specific port no(like 80) etc. In certain cases the categorization needs to go one step ahead and analyse the various sub-categories of a specific category as well. For example, in port 80 there may be more than one application running. And each application needs to have a separate categorization in order to observe their characteristics and optimize them later on.

The parameters required for bandwidth categorization is quite important based on the type of usage of the WAN links in a particular organization. Normally categorization based on applications is considered quite sufficient, and if not, there might be some additional categorizations based on the above parameters too.

Bandwidth Analysis:

Once the WAN traffic is categorized and the parameters for them defined, there are various metrics that can be employed to analyse the bandwidth consumed. This analysis would be used later on to optimize the bandwidth across WAN links. So, some of the parameters employed by various vendors to analyse bandwidth sent across WAN links are given below:

¤ Bandwidth usage levels per ip address, host or subnet.
¤ Top few Applications that are generating the bulk of the traffic.
¤ Bandwidth wasted due to retransmissions, dropped packets, aborted TCP connections by the server etc.
¤ Applications/ Users/ Servers that get the most traffic and send the most traffic.
¤ Identification of clients, network and server with the slowest/ fastest response times.
¤ Connection speeds and packet size histogram data.
¤ Largest and lowest number of simultaneous TCP connections at any given point of time.

Those were some of the performance monitoring parameters that can be tracked in order to optimize the WAN bandwidth. Some vendors might also give a graphical representation of various parameters for easy analysis either for the whole link or a portion of the link like a particular server/ application or URL etc.

Bandwidth Optimization:

Based on the traffic analysis, some of the following steps could be taken (depending on the bandwidth optimization vendor) to ensure that the available bandwidth is optimized across the WAN link.

¤ Virtual Links: The WAN links could be subdivided in to smaller partitions called virtual links. Each virtual link could be restricted to a certain application, user group etc. The advantage of sub-dividing the available links is to ensure that each application gets some minimum guaranteed bandwidth and that one application or user group does not occupy the entire/large portion of bandwidth available with the WAN link. Unused bandwidth is always available to others. Some vendors also allow dynamic creation of virtual links based on certain events like real time applications requiring more bandwidth suddenly, allocation of more bandwidth per user when the ERP applications are using less than a certain bandwidth threshold, etc.

¤ Policies: It is important to set priority levels to each type of application/ user group etc(In a scale from 0 to 10, for example) in order to make sure that mission critical traffic types like ERP, voice, video and others get a higher share of bandwidth than peer to peer applications, music downloads etc. which need to get a minimum share of bandwidth as long as the other critical applications are active on the WAN link. There could also be a limit to the minimum and maximum bandwidth that an application could use, for example. Certain type of applications could also be blocked by giving a very limited bandwidth or zero bandwidth (personal applications like youtube videos, movie streaming etc). Some applications (like emails etc.) are not set any priority level and go by the default average priority.  Certain vendors allow automatic creation of policy tables when the most important applications/ user groups are classified and assigned a certain priority based on their classification and analysis.

¤ TCP Rate Control/ Admission Control: When TCP packets arrive at a faster rate than that can be processed, these bandwidth optimization technologies could apply TCP rate control to the WAN links and inform the sending stations to send packets at a slower rate as excessive TCP packets need to be discarded and re-requested. Certain vendors also allow to manage bandwidth by measuring the speed of such TCP transmissions and adjusting certain parameters in the receiving end. Admission controls involve creation of policies to handle the additional sessions during a bandwidth shortage like blocking them or reducing the bandwidth allocated to all other/ certain applications by a uniform rate to accommodate these additional sessions etc.

excITingIP.com

In case you have any questions, you could contact us using the contact form or leave a comment below. You could also participate in the discussions in the Forum.

What is a Link Load Balancer?

As consolidated data centres are becoming more popular, the WAN links from the head office and branch offices/ remote sites have become very crucial for business. So, companies are buying bandwidth pipes from different service providers for double the capacity (or less) just to make sure that in case there is an outage of service from one provider, then they have some back-up. Sometimes such back-up links are not used much and the fail over, if the primary link fails is generally not instantaneous.

So, instead of having all the incoming links from the service provider(s) separately terminated and routed to, in to an organisation, a Link Load Balancer is used to terminate all these incoming lines at the gateway level and provide a uniform interface for the systems to connect to it.

Advantages of Link Load Balancers:

If there is an outage with a particular service provider, all the users/ active sessions are seamlessly transferred to the other active lines. The lines provisioned for back-up are also used by the regular users as the link load balancer now provides a uniform interface for all the systems to access any line. That gives some additional bandwidth capacity to the users. Bandwidth allocation, and bandwidth shaping can be done at the WAN gateway level using these link load balancer so that the critical applications always get their share of bandwidth and the chatty non-critical applications do not take up the entire bandwidth. This approach also enables an organization to route certain users to certain lines based on the best performing links to a certain destination (or least cost routing, application type, application priority etc).

Additional Features of a Link Load Balancer:

Certain Link Load Balancers also provide VPN and data compression services from branch to the data centre / HO. They also allow multiple types of WAN links to be connected to them (Like MPLS VPN, Leased Lines, Internet Leased Lines, Broadband connection, 3G Mobile through USB, etc) but this depends on the vendor and the model. Two such appliances can work in High Availability mode – in case one of them fails. Some of them also allow to use each link up to a certain maximum bandwidth (as the charges for some carriers are more when usage exceeds a particular limit). QoS mechanisms help set priority for certain types of traffic (like video, voice etc) which are critical and delay sensitive.

When a link is down, the users are accommodated in the other links and a notification is sent to the administrator via email or cellphone, as most of the Link Load Balancers can integrate with Syslog servers or SNMP network management software’s.

excITingIP.com

In case you have any questions, you could get in touch with us via the contact form or leave a comment below. You could also participate in the discussions in the Forum.

Cashing has always been used in many processes outside WAN optimization too. Network devices and computer processors cache important data so that they need not always be retrieved from the memory every time. If you are accessing this page through a web-browser, and you return to this page after some time, there is a good chance that the page will be re-loaded from your browser cache and a request may not be made to the original server where this content is stored. But some browsers do check if there has been any change in the content, with the servers before loading them from their cache.

Object Caching:

In a big organization, there are many files and templates that are frequently accessed by a large section of the employees. So, when a company has a centrally consolidated data centre, there may be a large number of such requests going to the servers in the data centre to access data. For example, all the users might be accessing the corporate website. So, every time there is a request, the web page needs to be downloaded from the servers in the data centre which takes more time and bandwidth. WAN Acceleration devices sit at the edge of such enterprise networks and analyse the requests that goes from the network. If it finds that a large number of users are requesting the same content (or parts of the same content is similar), then it stores that content (or parts of it – like a logo picture for example) in its local memory so that it can be delivered to the users in the LAN whenever they request for the same. This process is called object caching. This is also called proxy caching and useful for accelerating access to http, https and ftp content. Of course, if the requested object is not there in the cache memory, then the request is forwarded to the server. One huge advantage of object caching is server offloading – lesser requests to the server, in addition to the time and bandwidth saved.

Byte Caching:

Byte caching is another method of caching that usually has a pair of caching appliances working together – one at the branch level and another at the data centre level. Information streams are stored at either end and when ever a data segment being sent or received is similar to another data segment that is already present in the memory of both the cache appliances, that data stream is removed and a small tag is used in its place which the other appliance can recognise and replace with the original data stream. Some advantages of this method are: It works in the transport layer of the OSI model and hence is application independent. It just looks for full or part of data that is repeating itself, irrespective of the application or protocol. This works both ways for WAN traffic optimization – user downloading data from servers and also users uploading data to the servers.

A combination of the above two types of caching could also be effective.

Challenges facing Caching mechanisms:

The obvious problem associated with object caching is the frequent updation of website content. The home pages of most of the websites keep changing with time. Even some of the individual pages might change after some time. So, a proxy may need to check with the server if there are any updates for a particular web page requested by a user. And if there is any change, the new content needs to be downloaded to the cache memory and then given to the user. If there is no change, then the previously stored content may be sent to the user and the whole process might be limited within the LAN. This does increase the speed of loading of web pages and reduce the bandwidth required to load such web pages but it works only for those web pages that are frequently requested by the users. But there are many web pages that are frequently requested by many users of an organization.

There is another approach used by the caching appliances. They check frequently with the servers (even if a user is not requesting any content) for any updates/changes for the content that has been stored in the caching appliances. While this makes it faster for the users, the frequent checking for updation would itself consume a lot of bandwidth and create an additional resource crunch.

The storage required for caching such frequently required data (In addition to web pages, files, logos, images etc. are also frequently accessed from the central data centres) is also quite limited. So, there is always a limit to how much such content can be stored in the caching appliances. The content which is not frequently accessed or which is stored for a very long time might be discarded automatically. There are algorithms which determine which content to store and which content to discard based on frequency, longevity etc. There is also a consideration of the type of system that is used to index such stored data. Some caching appliances use file systems but some of them use object access through a hashed table in RAM which would enable any object to be obtained in a single read.

excITingIP.com

In case you have any questions, you could contact us using the contact form or leave a comment below. You could also participate in the discussions in the Forum.

Ever wondered why WAN optimization is required in consolidated data centre scenarios? Or how much difference WAN optimization makes in such situations? This article gives an overview of the techniques that are applied to optimize the data that is sent across the WAN network in order to reduce the bandwidth consumed by them and also to reduce the delay associated with WAN links.

What is WAN Optimization and why is it required?

Wide Area Networking has enabled a lot of applications (both for connectivity as well as optimization). We can today use the Internet Leased Lines and MPLS VPN Connectivity for connecting different branches of an organization. These are quite cost effective when compared to IPLC or dedicated Leased Line connectivity. One of the consequences of the affordability of such higher bandwidth and connectivity is the consolidation of servers and core networking components in a single data centre. This has definitely reduced a lot of cost (when compared to individual servers that needed to be replicated at each branch, in a few cases) and enabled applications like Virtualization (Where the resources of a server are shared between multiple applications as there is always umpteen processing power that is not utilized at all points of time, for example).

But consolidating resources through Wide Area Networking also comes with its own problems. There is a delay or latency induced in the packets that has to travel all the distance. This delay might also be due to the line conditions, amount of traffic at that instant, hardware induced delays like buffer delays both at the local end and ISP end. And there are always real time applications like voice, video etc. that travels along with bandwidth intensive applications like email and ERP that might cause un-acceptable delays if proper QoS policies are not set.

So, all these factors plus a lot more creates the need to have WAN optimization processes in place. Although the WAN optimization techniques do not eliminate the delays, they reduce it to a good extant. And WAN optimization induces its own security concerns but fortunately WAN optimization techniques address security in a much broader sense due to their unique position in the enterprise network where all the packets need to pass through such devices.

WAN Optimization Techniques:

Protocol Substitution:

¤ WAN Optimization devices manage chatty protocols inside the LAN and then encapsulate and manage communications across the WAN using less chatty protocols. For example, TCP (Transmission Control Protocol) requires acknowledgements for transmissions received and includes other mechanisms to do windowing, congestion management, slow start, etc. to manage its performance characteristics which induce further delays. Such TCP packets are often repackaged inside UDP packets for WAN transport or applications are re-architected to replace TCP with UDP for WAN Optimization.

¤ Multiple long messages are consolidated in to one or more longer ones. Each protocol has its own size of packaging messages. Video packets for example are longer than http packets. If the packets are small, they need to be sent and acknowledged more often and delay sensitive packets (which are longer) might have to wait. So, it is more efficient to combine a number of such small packets and send them in one go to reduce the number and frequency of such packets to traverse the WAN Link and to enable designers to choose optimization techniques uniformly for such WAN friendly message structures.

Data Substitution/ Cashing:

WAN optimization can look for recurring strings and data elements at the application layer. If these elements are copied across the WAN Link once, and they don’t change thereafter, a couple of WAN optimization devices can index them and exchange only index values between them so that frequently sent data elements need not be sent again and again through the WAN Link. For example, if a website like the company website is accessed by many members in an organization frequently, then most of the data of that website could be stored locally in a cache and sent to the employees instead of contacting the remote web server other than transactions which involve some real time processing like authentication etc.

Data Compression/ Encryption:

A variety of compression techniques are used to compress the WAN traffic. Header and Payload compression techniques use pattern matching algorithms to identify short, frequently recurring byte patterns that can be replaced by even shorter segments to reduce the final transmitted sizes. Different techniques might be employed to compress data across the WAN link but data which are already compressed like ZIP files or H.264 video packets etc. cannot be compressed much again. The traffic that is sent in the WAN links are often encrypted for security reasons as public networks are more and more utilized these days.

excITingIP.com

In case you have any questions, you could get in touch with us using the contact form or leave a comment below. You could also participate in the discussions in the Forum.