HARVEY SPENCER ASSOCIATES Effective Capture is the First Step in Digital Transformation By Harvey Spencer, President HSA inc. 2017 HSA Inc. Page 1 of 12
Background Digital transformation with SAP is not just a buzzword. In order to effectively compete in a global fast moving economy, all businesses have to transform their methods and processes. The promise is better competitiveness, happier customers, fast efficient processes and analytic tools for better management. But change is challenging as we convert from legacy systems to a digital economy. Receiving and capturing information from paper, email attachments, fax and electronic transmissions continue to present challenges to all organizations. Still this is often ignored as companies focus on streamlining the processes of electronic data and on new applications. The problem is that without a solution to automate the conversion and understanding of unstructured and semi-structured inputs including legacy ones, manual data entry becomes a huge business burden. This is not only a cost at the front end, but missing or late information leads to exponential cost, risk, and bad decisions downstream. Digital Transformation promises to eliminate these paper-oriented inputs. And although declining in use as an overall percentage of transactions, the fact is that paper and paper-formatted documents continue to be a major method of transactional communications between businesses. Sometimes despite the acceptance of digital signatures, this is because a wet signature is still required on the paper (see Certificate of Insurance below). But even when the physical side of paper is eliminated through PDF s and digital signatures delivered via email the problems of processing remain. The numbers are large and processing is expensive. Worldwide we believe that around 50bn transactions a year need to be converted a total of around 6tn characters to be captured. As one data point, the US mail shipped over 61bn pieces of first class mail in 2016 much of it is business related. While this has been declining, it is worth noting that the rate of decline has been slowing. Depending on the application and country, 80% or more of external communications is still paper based. With an average manual data entry cost of 63 cents, the cost is enormous. In the case of invoices for example, we estimate that over 5bn sheets of paper are sent out every year in the US from companies with over $5m revenue and there will still be over 4bn by 2023. Including 2017 HSA Inc. Page 2 of 12
PDF invoices, which some call EDI, then the volumes are much higher, but the processing challenges remain. Most of this information is manually keyed. An average invoice form contains around 120 characters of information that must be captured into the SAP system. At an average wage rate in western countries of $30,000 a year, that costs around $37,500 a year to capture the information from just 50,000 invoices 4,000 a month. So in the US alone, we are likely spending over $4bn a year keying information from invoices. Adding Europe with a similar sized economy likely doubles that with the rest of the world additive. But when we look at all the different departments and areas within a typical company using SAP, many hundreds of thousands of forms and other information must be key entered at a cost of several hundred thousand dollars a year per company. Paper formats are used in Purchasing; in Orders; in Contracts; in Shipping; in Human Resources in Sales and Marketing and in many other areas. In Banking and Insurance they are used for on-boarding new customers and loans or for filing claims. Much of this is manually keyed. Not only is this expensive, it introduces risk and it is time consuming. To solve the problem companies such as Open Text have developed advanced capture technologies to dramatically reduce the costs of extracting, validating and updating the SAP core systems, make it faster, more accurate and provide the ability to capture additional information effectively without cost. Speed of processing is important and is one of the benefits of going digital, but if legacy input systems cannot keep up, the advantage is lost and more nimble companies will overtake. Accuracy is critical as systems built to use in memory real-time systems such as S/4 HANA do not give you time to correct mistakes before the process processes are likely to have already been put in place and decisions made. Undoing these becomes expensive and adds to risk. Reducing Cost & Faster Processing These systems are designed to reduce the cost of capturing information from incoming forms on a broader basis by leveraging advanced Optical Character 2017 HSA Inc. Page 3 of 12
Recognition (OCR) and other recognition capability combined with AI based machine learning systems. Utilizing identified keywords to locate the needed information, business and validation rules associated with that information are then applied. These are then combined with a learning system to continuously understand what the variations in the form are, and learn what clerical staff do in order to build the automated extraction and validation systems. In the case of Open Text, this all operates within the user s SAP systems, whether on-premise or in the cloud as far as the user is concerned. The approach uses OCR to convert the whole document set to text. Taking the illustration below of a Certificate of Liability Insurance, the system ignores boilerplate text and looks for key words such as Insurer and Insured as well as the tabular amounts insured for and the signature. It finds the check off boxes such as type of Insurance and liability. In the case of an invoice it looks for the key information including Invoice Number, PO Number and Date etc. As you may see the Certificate of Insurance example was scanned from a printed FAX. The information can be difficult to read and the paper may have 2017 HSA Inc. Page 4 of 12
been angled or distorted. Today s image enhancement and improved OCR deals with most issues if it is legible to a human the machine can read it and in some cases the machine can outperform a human. But in the case of FAX s, problems caused by the sending machine can increase if you receive a fax on paper, print it and then scan it. Sending directly from a computer system and receiving on a fax server or converting to a PDF and delivering with an email helps substantially because lines are straight and characters are well formed and cleanly separated. The system then uses key words to locate the information or box next to that information and extract it, tagging it usually with XML tags that can easily be imported. This was previously impractical due to speed performance constraints. The system also identifies other elements such tabular line item information where the details such as coverage or in the case of the invoice quantity, product ID, cost and extension is laid out. Set up time for new document types becomes greatly reduced from previous methods and when a new format or document type is encountered, the system may be able to automatically adjust if the form is similar in layout, language and/or terms to a known previously captured form. But if it is a new type of form there is a need to adjust the proposed zones and add rules. New capability leverages previous knowledge and combines that with the operator s activities and knowledge to automatically set up and refine the extraction and validation process. One challenge has been extraction of tabled information such as line items on an invoice. Line item extraction is not easy as there are many variables. A simple case is one item equals one line typically a quantity, description, unit amount and extended amount (see figure below). In this case it is fairly easy to extract each line from the table, balance to the extended amounts and the totals. But invoices and other forms frequently are not as simple as this. In the case of an invoice, item descriptions may take up multiple lines which vary according to the descriptions. They may even carry over from one page to another with a break point wherever the output system decided to put it. Totals and where and how discounts have been calculated as well as taxes have to be 2017 HSA Inc. Page 5 of 12
understood, validated and extracted for posting. To be successful systems have to deal with many variations. Sometimes, invoices consist of many pages such as lists of phone calls on a cell phone bill where differing calls may need to be allocated to differing ledger accounts. Manually going through these is not cost effective, so many do not bother. But capture can allow you to correctly capture and allocate these items. The same situation occurs with orders and payments. Check and list as Wholesale Remittance systems are called in the US, consists of a payment advice and maybe a check (cheque) if not a direct transfer, followed by a multi-page list of all the items being paid. Manually going through all these items is not always cost effective as each line item must be entered but capture systems can quickly dissect the document line items and based on rule sets and synonyms automatically code each item for automated posting. Electronic PDF invoices need a somewhat different approach. Many companies are sending attached PDF s by email to dedicated addresses in order to lower costs of paper, shipping and convenience as well as improve speed of delivery. PDF files attached to emails present somewhat different challenges to the capture solution. A PDF is basically a wrapper, which allows a PDF viewer to render the information into a human readable format. Inside the wrapper may be formatting information followed by text or it may be an image or it can be a mixture. The reader of the information usually does not know what lies behind the viewing. So Capture solutions must deal with this. If the information is textual, as is usually the case when created by an accounting system, then the capture solution can remove the formatting information, but it still has to identify the needed fields and validate them the only piece that gets missed is the OCR. Once the data is extracted then the same rules and validation must apply. But if it is an image or even a mixed text and image PDF, then it must be OCR d which can be done as well as textual extraction and the results compared. 2017 HSA Inc. Page 6 of 12
Most departments in a company have incoming semi and unstructured information that they must extract data from. There are many examples of common forms that we have to process on a daily basis. Even companies that have moved to Concur have to deal with receipts where cash was used. These contain itemized line details that need to be analyzed, balanced to the expense report and approved before posting. Companies need to extract information from sales orders or order confirmations. And documents such as applications, signature forms and contracts all of which can be converted with key terms and phrases extracted and highlighted for indexing, posting or analytics. Exceptions can be highlighted. Improving the set up and broadening the types of forms that can be handled, can still require some presorting of inputs to classify them and a dictionary of previously known tags. If the system comes across a wording or tag that it has not been trained on, it does not know what to do. The user operator then has to become involved. Integration Improves Accuracy and Speed It is important to create a connection to the relevant business process in order to add context. Capturing the information is one thing, but guaranteeing its completeness, accuracy and relevance to use it efficiently without human effort, is a separate issue. By integrating with the SAP transactional and systems of record the capture system adds context the incoming document data. It allows for validation, correction, formatting and tagging. It may need to add content from the relevant SAP system and changes may be needed. Often terminology and tags may be different for the sender so the recipient s capture system must translate terminology to make it understandable to the SAP system In a typical environment, many transactions are reactive and can be referred to and validated within the SAP system. For example an order may be received by sales as a FAX or PDF attachment to an email (see figure below). The system will OCR it, validate against basic rules contained in the capture solution and with more advanced rules in the SAP order entry system. Once validated the SAP workflow prepares the order for approval, 2017 HSA Inc. Page 7 of 12
which is usually reviewed by the manager. The order is then completed (posted) and the confirmation sent. If there are errors or a problem, the order can be routed back to sales or maybe the sales order department contacts the customer. Other examples are similar from a technology standpoint, but different in their terminologies, validations, and workflow. A quotation may be sent in via emails or fax as a result of a purchasing request for quotation; a resume may be sent in response to an advertisement that needs key word extraction; an expense statement needs receipts to validate approved expenditures; a claim can only be valid if sent from a policy holder, a new loan application usually has an account holder. An order results in a shipment and an invoice. All these examples create a situation where there is a record or a transaction that can be used to validate and if necessary, correct the incoming information. But the reference numbers on the incoming form may not be the same as those in the SAP system. For example part numbers may not be the same as in the purchase order; the discount taken on a payment may not be applicable; the shipment might have been damaged or partial by integrating with the SAP record, these can be cross referenced and corrected. Very occasionally the OCR may get a wrong answer that cannot be selfchecked. For example due to noise or bad printing it may confuse a 6 with an 8 or vice versa. It could confuse an a with a b if the vertical line was missing. Frequently this can be auto-corrected from the SAP system of record. 2017 HSA Inc. Page 8 of 12
Accessing and balancing to the source of information reduces inaccuracies and improves performance Straight Through processing Integration allows for straight through processing in some cases without human review! Payments can be faster taking advantage of discounts and allowing for faster closing. This can happen if here is a correct purchase order number on the invoice, which the system can use to validate the amounts against the PO, and the actual receipt from the goods receivable department s records matches. If it all balances then the invoice gets approved for payment and posted into the SAP ledger. Even if there are mismatches, the system can discern the problem and out sort the invoice for a manual review. There are many areas where this technology helps keep up with today s digital transformation challenges. Faster processing can provide advantages. As an example with A/R side, reconciliation and quarterly closing can be sped up through matching incoming payment advices to invoices in the receivables system and then matching to the bank statements thus improving and speeding up reconciliation. Capture provides more information for analytics. Most systems that rely on manual key entry even those who are looking to reduce costs with offshore processing -- enter information into a fixed form template. They ignore any information that is not relevant to the transaction purpose that they are engaged in. But because capture systems capture all the information contained on a form adding relevant metadata, the user can leverage additional information as needed. Better Usability From a usability standpoint, by integrating with the back end ERP accounting system, users are more comfortable with the system as they are not presented with a new set of screens to understand. There has been a move by SAP towards simpler front ends such as FIORI, which is being supported by companies such as Open Text. This supports users as they move to simpler HTML5 based screens capture systems but for those that are using 2017 HSA Inc. Page 9 of 12
traditional complex SAP user interfaces it is much easier for first time users to work with a system that looks familiar. But the real value is in the ability to achieve a higher level of automation quickly. Extracting key information accurately from paper and other unstructured inputs effectively frequently slows up decision making while users key in the information. Capturing this information more quickly is critical to making fast decisions. A fast in-memory database such as SAP HANA can deliver intelligent business decisions, but to leverage its full capability information must be complete. Advanced capture with tight integration provides this. What s Next? Mobile capture is a fast growing area as mobile workers like sales, need to capture and send in information. Today s capture solutions are making that much easier with use of the mobile phone capture hovering over a form type until it is automatically captured The industry is increasingly concentrating on improved UI and UX and human assisted auto learning and the newer systems learn from the user. Starting with a basic known form type like an Invoice, the AP clerk can review the suggested results and if necessary make modifications the system learns from this and the next time an invoice from that supplier is identified, the system automatically knows what to do. In this way we can reduce set up time to being part of a clerk s job he or she is in effect training the robot. This eliminates the need for expensive dedicated IT staff who first have to learn how to set up and configure the capture system and then have to learn what data needs to be extracted from which document. As companies transition from in-house systems to cloud services cloud based capture is becoming increasingly important. The advantage is that effectively unlimited compute power can be applied to the problem and so the customer starts to control levels of need and pay accordingly. Capture in the cloud seamlessly integrates with SAP cloud based systems. In the case of Open Text the OCR, and maybe some other activities, may be processed on the Open Text cloud servers, but the user does not see this. He is still interacting and controlling the system through his SAP system. 2017 HSA Inc. Page 10 of 12
The Future The ideal situation is to be able to automate the input of information into the business systems regardless of what the source is and what the format is. If it is understandable by a human then it should be understandable by a computer. If the human knows what to do with the input, then the computer can be taught to do the same thing. Advanced learning systems can start to figure out what the process, what information is needed where and where to get ancillary information. But we are a long way from that. Inputs today consist of many different elements and formats. A human looking at a piece of text will utilize a mix of experience and intelligence to decide what it affects and what information to extract from it. He or she may need to refer to another piece of information that may not be an obvious match. He may need to refer to a colleague or ask management for assistance. Improving Classification is a start. Currently there is an assumption that incoming documents are pre-sorted. This may occur through the use of a specific address or box number for say all invoices or it may have to be a manual process. But exceptions like for example attached correspondence -- can slow the system. Being able to automatically identify the incoming information and decide what is the important and relevant data further reduces cost and improves process. We are now starting to incorporate such AI tools as language translation so that we can process foreign invoices or orders automatically; semantic understanding in order to improve customer service by interrogating incoming correspondence; and sentiment analysis to better manage customer relationships or gain a better idea of a new job applicant. We can even start to predict outcomes based on the inputs. Being able to interpret more data elements becomes critical. As mobile information affecting day-to-day business transactions ever increases, data elements especially unstructured data is increasing exponentially. Cloud based systems assist with the mobile workforce. Capture solutions are evolving to operate in this environment and they learn continuously. 2017 HSA Inc. Page 11 of 12
Summary. Capture is a critical component for companies to be successful in their digital transformation. Even millennial companies receive unstructured human understandable inputs and are often inefficient when dealing with them. The fact is that we will always receive much unstructured inputs including paper and probably FAX that must be understood in as close to real time as possible. email attachments do not solve the problem of interpretation and extraction. This will continue to be an expensive challenge for companies. Tight integration with the SAP system is critical as it provides context. Faster in-memory real time back end systems such as S/4HANA have been developed, but often the front-end capture holds up the process. What is the point in having real-time processing if you do not have all the information you need to process? Capture has evolved from simple extraction of data from fixed, known forms, to variable forms that relate to a business process. Configuring systems by the user showing the software how to operate on the data are starting to be deployed. Machine learning builds the capability for a future where all human understandable inputs are understood sufficiently for systems to be fed usable tagged data. Doing it the old-fashioned manual way is expensive and can lead to a lack of competitiveness. Today s capture systems are built to deliver fast accurate data from traditional human understandable paper based systems into real time databases -- without this, business cannot keep up. Traditional manual input is not an option. 2017 HSA Inc. Page 12 of 12