This is not the document you are looking for? Use the search form below to find more!
ERP Software Testing and Its Quality Evaluation System
0.00 (0 votes)
Document Description
ERP Software Testing and Its Quality Evaluation System
File Details- Added: April, 25th 2011
- Reads: 134
- Downloads: 0
- File size: 152.82kb
- Pages: 11
- content preview
- Name: jelle
We are unable to create an online viewer for this document. Please download the document instead.
Related Documents
Operational Route Accounting ERP Software: Benefits and Obstacles
by: weightsalt96, 2 pagesERP application vendors know that there is a even bigger slice of the pie out there for them
White Paper: Outsourcing Quality Assurance, Outsourcing Software Testing, Software Development Methodology
by: georgina, 1 pagesWhite Paper: Outsourcing Quality Assurance, Outsourcing Software Testing, Software Development Methodology
Infosys - Business Process Scalability | Independent Testing and Validation
by: manisha229249, 9 pagesInfosys white paper on driving business scalability through modern software quality. Read Infosys white paper"Infosys - Business Process Scalability | Independent Testing and Validation"
Software Testing: |Art - Practice|=? Aditya Mathur
by: auxiliadora, 12 pagesSoftware Testing: |Art - Practice|=? Aditya Mathur Purdue University Department of Computer Science SERC Showcase Ball State University Muncie, IN November 11-12, 2009 Objective ...
Importance of Software Testing for successful IT Management
by: markwaltzer, 1 pagesSoftware is a series of instructions for a computer to perform a particular task or a program. The IT industry may be flooded with a variety of software testing services. Software testing company ...
ISO 13485 as a Quality Management System for Medical Devices
by: globalpanel, 1 pagesThis presentation will help you to understand ISO 13485 and implement a successful quality management system, comparing it to ISO 9001 and FDA & European Union requirements.
Software Testing for CSPs - Market Analysis
by: ronethell, 5 pagesOur analysis gives never before published market information including trends, drivers, market sizing and competitive analysis in the software testing segment based on direct customer and supplier ...
Electrical Testing and Tagging
by: testandtag365, 4 pagesWe provide electrical testing and tagging services, at best prices with high quality customer service in Melbourne.
Intermediate Microeconomics and Its Application Nicholson 11th Edition Solutions Manual
by: gordonbarbier, 48 pagesIntermediate Microeconomics and Its Application Nicholson 11th Edition Solutions Manual
Linear Algebra and its Applications Lay 3rd Edition Solutions Manual
by: georgesheslers, 48 pagesLinear Algebra and its Applications Lay 3rd Edition Solutions Manual
Content Preview
- ERP Software Testing and Its Quality Evaluation System Chen Xuesong1 Beijing Ufsoft Co. Ltd., Beijing, P.R. China 100085 Sun Baoqian Beijing Ufsoft Co. Ltd., Beijing, P.R. China 100085 Shao Kai Beijing Ufsoft Co. Ltd., Beijing, P.R. China 100085 Abstract The software testing techniques for ERP software accompanied with the establishing of a quality evaluation system accordingly is discussed in this paper. We try to provide a comprehensive and generalized ERP software quality evaluation system by analyzing the quality affecting elements within which the evaluation points are derived. For each evaluation point, the testing items are interpreted separately. Such a quality evaluation system can help both software vendors and users evaluate the quality of an ERP software product under delivery. Introduction The idea of Enterprise Resource Planning (ERP) has experienced an evolvement from a fashion to a rational requirement. ERP projects have been widely implemented or begun to implement in thousands of diverse enterprises and industries for the purpose of improving the finance, commerce and business managements of the enterprises. The ERP software products are playing a primary role within the ERP projects. They are designed according to the business goal and intending to improve the automated management of a target enterprise. The ERP software quality is seriously affecting the implementation success rate of an ERP project. Unfortunately, it was reported that the implementation success rate of the ERP projects keeps rather low level due to various reasons. The objective causes may be the lack of abundant startup funds and the variable business activity, which results in the inconstant requirements. The most critical reason may be the lack of sufficient software testing against the requirements of the target enterprises, according to which the quality of the ERP software can be evaluated when delivering it. In the face of the requirements from different industries, such as banks, securities, insurances, and etc, the ERP software vendors have to develop the customizable software product components that integrated in a large-scale ERP product to satisfy the diverse requirements, which results in the high complexity of a product. What points are to be tested, how to ensure its quality, and how to guide the vendors to evaluate the quality before it is released to the market or its delivery to the users, are 1 Chen Xuesong, Enterprise Post-Doctor Co-educated by Computer Science Department of Computer Science and Technology, Tsinghua University.
- hard problems that bothering the vendors for a long time, almost all vendors are facing this situation. We are devoting to develop a kind of ERP software quality evaluation system 1 for the different scaled ERP software vendors, in order to help them test the software products sufficiently and assess their quality objectively before delivery, and to help the target enterprise to accept the ERP software product by a referable and feasible guidance. Hereby, we hope to promote the implementation success rate of the ERP projects. This software quality evaluation system involves a set of elements that affect the software quality. Due to the pages restriction, we only interpret the part of software testing elements in this paper, which is one of the most critical elements of evaluating the software quality, and necessary to the ERP software vendors at present. 1 The problems and requirements In recent years, despite of the global economic depression, the ERP projects are still sharing large market quotient accompanied with the increasing sale performance of the ERP software products. Ensuring high software quality, promoting the delivery capability and reducing the delivery cost are the core problems concerned by both the software vendors and the target enterprises. ERP software vendors are devoting to provide the user-satisfied products continuously. An ERP software product is generally a large-scaled distributed software system, which may integrate thousands of software components, functions, business operation-flows, and often is operated via the INTERNET/INTRANET 2 . System engineering methodologies are often adopted to evaluate the quality of a product to be delivered. It starts from analyzing the characteristic of the ERP software itself. Unfortunately, few documents and papers introducing the quality evaluation system and those are adaptable to different-scaled enterprises are found at present. The fact that reflects the developing state-of-the-art of most ERP software vendors may include the frequently variable requirement, tight developing schedule, and, most of all, insufficient testing before the delivery, which results in a software quality evaluation system guiding the test ahead of being release to the market is very necessary. Facing a submitted large-scaled ERP software product, the users of the target enterprises find it difficult to make a decision on “whether to accept this product?” or say, “whether the product can satisfy our needs?” Thus, the software quality evaluation system can also help the users determine whether to accept or to refuse this product. 2 Elements Affecting the Quality There may be a lot of elements affecting the quality of an ERP software product, including:
- ? Specific requirements from the target enterprises of a given industry; ? Product application model designed for the target enterprises; ? Software development model selected for the product under developing; ? Software processes undergoing; ? Testing sufficiency; ? Documentation system & Training system & Service system and its standards, etc. The testing sufficiency seemed to be very important and critical to the quality of a software product to be delivered from the intuitional point of view. A basic element of evaluating the quality is whether the operation-flows designed into the ERP software have covered all the required business operation models of the target enterprises, and they are suitable to other enterprises with a little customization. This kind of consideration may result in a realistic designation for an ERP software product with given resources. Furthermore, the operation-flows must leech on to an application model designed for an ERP software product. The application model is often designed as a distributed architecture, in respect that a contemporary enterprise may generally include a group head office and some branches. The software lifecycle model adopted for relief the effect of the requirements variation is another element affecting the software development quality. It evaluates that if the developing schedule arrangement is appropriate for the developing team to ensure that the necessary requirement analyzing, designing, coding, and every level of the testing including ?, ? testing and the acceptance testing can be deployed. Whether the necessary software processes for each product development phase in lifecycle have undergone affects the development quality very much. The quality of the whole product can not be guaranteed, if one or more key processes of the product lifecycle are not managed effectively, and the product development status may fall into confusion. The integrated documentation system, training system and service system can also affect the quality of an ERP software product. They are regarded the necessary accessories the software vendors must provide released with the software products and as an element of evaluating the service quality. The testing element is the most primary element of the quality evaluation system we are to interpret in this paper. 3 ERP Software Testing Contents A large-scaled ERP software product is often both a distributed system and a
- database application system, which is developed for ERP business application purpose. When testing the ERP software, these characteristics must be validated to be fault free as a running environment support previously, in accordance with the contract in which part the running environment quality requirements are specified. The compatibility of devices, operation systems, network connections and the application software are the primary assurances for the applicability of a product. 3.1 As a distributed system As a distributed software system, the stable connection among the server and client computers via many kinds of network devices is a basic requirement. The devices may include the routers, switchers, agent servers, hubs, modems and cables, and so on. On the basis of the stable connection, the data transfer rate and safety are two significant elements affecting the performance since the ERP software transfers very important business data, which are updating frequently and may be top secret against other competitors from the same industry. Although the reliability of hardware and database system may contribute to promote these performances, the middleware running within the server (or cluster server) and the client applications running within the client terminal are the key objects that affect the data transfer rate and safety. Here are some examples of pertinent connection modes: connecting via modems, network adaptors, etc., and via Internet, Intranet or LAN (Local Area Net), etc., where may involve data transfer on many kinds of network proxies. 3.2 As a database application system As a database application system, fast database access and safety data storage are the critical. A middleware connecting the client application and the database is demanded to be of high performance and database lock free while writing and reading data from a database under the circumstance of many user’s concurrent operations that often occurs among the group and branches in the target enterprise. Data access accuracy contains two aspects of meanings: the data in the database are stored accurately enough for further calculation, and the data fetched from the database are right the user wants to get within given time. If some data are fetched right, but time out, an exception may be thrown out. For some distributed database system, the data load balance allocation is involved to avoid data space exhausted. Multi-database concurrent access may encounter data access safety problem. The middleware is required to ensure the data accuracy from different kind of database system for different users solely or concurrently.
- 3.3 As an ERP software system The representative characteristics of the ERP software may be their anfractuous business operation-flows, which results in the functional testing is considered to be the most time-consuming development efforts. However, these functionalities are critical to help the ERP project succeed in the implementation. First, the application model of the product in the target enterprise must be realized, which is a critical test point. The application model may include the business transactions occurred between the group head office and sub corporations, or among the sub corporations. These business transactions are generally managed by the group head office in the target enterprise. The business data are collected and gathered to the account monitored by the group head office from the “Integrated Management” point of view. The following items are often considered while designing the test cases according to the characteristics of the ERP software: business path feasibility testing, values calculation, data precision testing, business data interface testing, concurrent operation testing, data access authority testing, short-cut key testing, print testing, template usability, customization usability, online help documentation testing and so on. These considerations are some common requirements of each product module that may contain the corresponding software execution circumstances. The following brief example 3 of an application model for an ERP software product, which only consists of a supply chain management (SCM) and a finance product, may be able to help interpreting the application model we evaluate the ERP software quality. Different enterprises may generate diverse requirements to a certain ERP software product according to their routing business operation-flows. To some extend, these operation-flows may generally involve some different products, such as an SCM product, which may include the purchasing, inventory, delivering, and the sale management business modules and etc., and a finance product, which may include a general ledger, an account receivable and/or payable(ARAP) and a stock management business module. Finally, hundreds of these operation-flows that implemented using these business modules of several typical products (e.g. SCM and finance products) are integrated together to form a large-scaled ERP software product with the function of being customized to satisfy the operation-flows variations from the enterprises of different industries. The business relationships and bill operation-flows of the routine operation in a typical enterprise are shown in Fig 1, where the arrows denote the business bill transfer directions. Fig 1 A brief application model example of an ERP software product In the example above, the following business test points may be included:
- ? In finance product: saving initial balance, making vouchers, signature/canceling signature, checking and approving vouchers/canceling, keeping accounts/canceling, deleting vouchers, settling account/canceling, updating vouchers concurrently or solely, the bill transfer from ARAP and Stock management module to General Ledger module with given data accuracy and so on. ? In SCM product: making purchase bills, sale order bills/sale invoices, entering /out of warehouse bills, inventory settle accounts bills and the data transfer to corresponding finance product module within given accuracy etc. Hundreds of business operations-flows can be generated from combining these business operations according to their occurrence sequences, which is the most complex and time-consuming efforts of testing an ERP software product, and have to be tested by simulating the actual usage in the target enterprises. Similarly, there may be many other products in the ERP software, which contain many modules either, and constitute a large-scaled ERP software product. The test managers are often asked to tell how many such operation-flows that have to be tested exist in the product and what is the business complexity of an ERP software product. It is a fairly difficult question to answer 4 . Although the MaCabe and Halstead methodologies 5 can help to calculate the complexity of a piece of program, they are difficult to calculate the large scaled operation-flows. Here, we provide a set of operation-flows calculation empirical formulas to answer the above question. An operation-flow can be considered as a series of Operation Pair (OP) jointed one by one from the end of an operation to the beginning of another operation. For a large-scaled ERP software product, the product modules and their final business node are carefully defined in designing phase. We can get the significant OPs from the related operations within each product final node, and then, extend to all the product nodes, and then, to all products. The software operation-flows that can be regarded as the business complexity of the software may be obtained by conjoining the OPs to a complete business path operated by the users practically. Thus, the business complexity of an ERP software product (i.e. the number of operation-flows) can be calculated by the OPs from the ERP product level to the product final node level hierarchically. Suppose that there are n product modules within an ERP software product, whose business complexity is N. We can get N1 related product pairs, i.e. the number of product pairs, such as General Ledger and SCM, SCM and Manufacture product 2 pairs, etc., and N2 independent product modules, where N1 Cn? N2 . Let Nci (i 1,2, ,n ) be the business complexity of each product module, the product business complexity N is calculated by:
- N2 N ?(N N1 ci • N c j ) + (? N ci ), i ? j, i, j ? {1,2, i =1 , n} Equ.1 Where the first part is the business complexity of those related product modules, and the last part is that of those independent modules. For each product module, the business complexity Nci , i 1,2, ,n, can be calculated from the business complexity N f i of each final node within the product modules by: m2 Nci ?(N m1 fi • N f j ) + (? N fi ), i ? j , i, j ? {1,2, i =1 , m} Equ.2 where m1 is the number of related final node pairs, m2 is the number of those independent final nodes, m is the total number of final nodes within each product 2 module, and exists: m1 Cm?m2 . The business complexity of a final node N f i (i 1,2, ,k) can be obtained by: k Nf i ?S i =1 opi , SOPi = Equ.3 where k is the number of the significant OPs within a final node. SOPi (i =1,2, ,k) is the conditional significant OP state at present parameter settings. An OP is the minimum business operation-flow unit within a final node, and the basic component of practical business path. The business operation-flows may consist of many these significant OPs with usable state. Hereby, we get the business complexity of the final nodes N f from summing the usable state OPs. i 3.4 As a software product From the software product lifecycle point of view, the necessary phase testing often includes, unit testing include the static inspection (e.g. code walk through) and dynamic code execution, union testing, integration testing, system testing (e.g. software environment compatibility), and users’ testing, etc. In addition, some special testing may be required according to the users’ requirements, such as the reliability testing and the maintainability testing 6 . For some serial products, the updating testing is fairly important. In the product updating testing, both the codes and their pertinent data in the database of the historical edition are needed updating for the consistency of
- the software. All tables in the databases are to be visited once at least before and after the updating to ensure the data accessing is available for the codes. Two kinds of typical testing, i.e., the functionality testing and the performance testing (including stress testing and concurrent operation testing, etc) have to be implemented within the corresponding development phase to ensure the requirements on functionality and performance. Large amounts of efforts will be paid to perform each kind of testing mentioned above, which may be one of the most time-consuming jobs that contribute to the developing costs greatly. In a software development organization, balancing the cost and profit against the schedule appropriately is a key problem to the dept. of software process management. Many development decisions are made based on such balancing. Thus, the following problem, how and how many test data should be selected and when we can state that the software is well-tested, can be resolved by considering the business goal of the software vendors, and balancing the cost and profit against the schedule, since most of the software vendors are of profit purposed 7 . Because of the constraints of human resources, time resources and development cost, most resources have to be assigned to test the primitive functions (or say, business operation-flows) that the users required mostly. The testing priorities of these primitive business operation-flows can be obtained from the statistical analysis on the users’ requirements based on the statistical testing point of view. The method of the expert system for prioritizing the business operation-flows is suggested, where the experts can mark the business operation-flows according to the information collected from the statistical analysis against the requirements and software process data. Some testing tools, developed for either the special product or all-purpose software products (e.g. rational robot), may help to reduce the efforts and shorten the testing period. A basic principle of software testing is to expose the critical defects early to avoid the delivery deadline being postponed due to correct these defects, such as the lower execution performance and the defects of requirements / designs. 4 Quality Evaluation System An ERP software quality evaluation system, developed on the basis of the testing discussed above can help both the software vendors and the customers understanding the state-of-the-art of the software quality under delivery. It serves like a mirror. From the mirror, the software vendors can see their developing capability and collect the information on how to improve the software process; the users can see how their requirements have been satisfied and how well the software product can help them to improve their resources planning. In general, all the testing mentioned above can help to collect the information for evaluating the software quality. Thus, the software quality can be interpreted in a
- quantitative way with the data collected from the testing. Based on the opinion of quantitative measurement, a series of standard software process activities directed by the mature software development technologies, such as software capability maturity model (SW-CMM) 7 , are put in action, data collected, and the corresponding quality evaluation and prediction can be deployed. Since too many elements are to be considered, evaluating the quality of a software product can be implemented from the System Engineering point of view. The quality evaluation system for the ERP software can be established from the following aspects: ? Functionality: to determine whether the software has implemented the requirements of the users, both functionality (required business operation-flows) and operation interfaces. The functionality evaluation aspect is implemented by the analyzing the information obtained from the functionality testing, such as the business operation-flows testing. ? Performance: to determine whether the software can response the users’ operation in time, including the network transfer rate, database access efficiency, the concurrent operation controls, multi-user mass data query and calculation, distributed data gathering, etc. This aspect of evaluation is implemented by the performance testing, such as the stress testing or database access efficiency testing, etc. ? Reliability: to determine whether the software is failure free while finishing the specific functions in a given environment within a given time period. The reliability evaluation is often performed by analyzing the software reliability models with reliability failure data. ? Implementation: to determine whether the software can be installed into the user’s hardware/software environment, established the necessary initial settings and data, and operated smoothly to perform the user’s required business operation-flows. The implementation quality is often evaluated by the feedback appraisal information from the target enterprise, and the implementation cost and period. ? Service: to determine whether the necessary after services, trainings, maintenance and documentation of latest product updating are available. The degree of user satisfaction of services is an important evaluation standard on the quality of services. Each part of the above evaluation aspects can be expanded to a series of sub items in detail, which involves the specific evaluation points accompanied with some specific technologies, e.g. reliability evaluation technology. Using these evaluation technologies can help the evaluator get accurate information of the software quality, and thus, figure out objective assessment, supply important quality information to both software vendors and users 8 .
- The evaluation systems of the functionality, performance and reliability aspects have been drawn for the development organization of a software vendor. Since not all items of this evaluation system are appropriate to different scaled software vendors, a tailoring guide of this evaluation system is also needed. 5 Future Works Our future work focus on the developing of a general quality evaluation system of the ERP software for different-scaled enterprises, followed by a tailoring guide used for helping these enterprises obtain their appropriate quality evaluation system of the ERP software products. 6 Summary & Conclusion The requirement of establishing a quality evaluation system for ERP software based on testing is discussed in this paper. The elements affecting the quality of an ERP software product are analyzed, and the testing that may be involved in evaluating the quality of an ERP software product is interpreted from the aspects of distributed system, database application system, ERP products and software product engineering point of view. The quality evaluation elements from which the ERP software can be evaluated are interpreted separately. The business complexity of ERP software is evaluated by giving a set of formulas. The quality evaluation system for ERP software is established for guiding the testing while the software product is to be delivered by the vendor, and guiding the user to make decision on whether to accept the product or not. 7 Acknowledgement The author was pleased to thank Beijing UFsoft (Group) Co. ltd. for providing the software organization environment and the contributions of ERP software testing practice. 8 References 1 JT-SQE (with X. Shi): A Software Quality Evaluation System, Journal of National Sciences, Vol 6, No. 1-2, 2001, pp.511-515. 2 Roger S. Pressman: Software Engineering A Practioner’s Approach[M], China Machine Press, Mar. 1999. 3 Zhang Yi: ERP and SCM, CRM [M], publishing house of electronics industry, Mar. 2002. 4 Rick D.Craig Stefan P.Jaskiel: Systematic Software Testing[M], Publishing House of Electronics Industry, Oct. 2003.
- 5 Zheng Renjie, Yin Renkun: An Overview of Software Engineering, Tsinghua University Press, Apr. 1998. 6 Michael R. LYU: Handbook of Software Reliability Engineering [M], Publishing House of Electronics Industry, Mar. 1997. 7 Yang Yiping: CMM methodology and its application [M], the people’s post publishing house, Apr. 2001. 8 Integrated Testing Under Web, http://www.51cmm.com/SoftTesting/No018.htm
Add New Comment