Ontology-Based Model of Law Retrieval System for R&D Projects Wooju Kim Yonsei University 50 Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea +82-2-2123-5716 wkim@yonsei.ac.kr Minjae Won INNOPOLIS Foundation 27-5, 123 beon-gil, Expo-ro, Yuseong-gu, Daejeon, Republic of Korea +82-42-865-8894 minjaessi@innopolis.or.kr Youna Lee Yonsei University 50 Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea +82-2-2123-7754 yuna2607@naver.com Donghe Kim Korea Railroad Research Institute 176, Cheoldobangmulgwan-ro, Uiwang-si, Gyeonggi-do, Republic of Korea +82-31-460-5483 kdh777@krri.re.kr HaeMin Jung Yonsei University 50 Yonsei-ro, Seodaemun-gu, Seoul, Republic of Korea +82-2-2123-7754 skymababsa@hanmail.net ABSTRACT Research and development projects have close relationship with laws. In some cases, new technologies resulted from R&D projects can t be used because some statutes restrict them. The reason of this problem is that researchers don t know exactly which laws can affect their R&D projects. To solve the issue, we suggest a model for law retrieval system that can be used by researchers of R&D projects to find related statutes. Input of this model is a query document that describes the main contents of a project. By using ontology, legal terms are extracted from the document and statutes defining them are retrieved as a set of related laws. After this searching process, statutes are provided to researchers with their ranks, which are assigned using relevance scores we developed. By using this model, we can make a system for researchers to search a list of statutes that may affect R&D projects, and finally, they can adjust their project s direction by checking the list, preventing their works from being useless. CCS Concepts Information systems Information retrieval Retrieval models and ranking Keywords Ontology; R&D; Law Retrieval System 1. INTRODUCTION In the long term point of view, the research and development process needs many steps from initial planning to final commercialization. In many cases, statutes related with new technology are revised or newly enacted during the period of its Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only. ICEC '16, August 17-19, 2016, Suwon, Republic of Korea Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-4222-3/16/08 $15.00 DOI: http://dx.doi.org/10.1145/2971603.2971629 R&D process. Changes in the related laws can affect the research direction, causing all the investment to be wasted. Namely, the developed technology can t be commercialized even though considerable amount of budget was spent for the R&D project. Along with the financial issue, the problem of losing opportunity to enhance the technological competitiveness in the worldwide market also exists. In order to deal with these problems, it is necessary to identify a set of related statutes in time. In this article, we proposed a model of law retrieval system to search a set of statutes relevant with a query document. For researchers, query document can be a R&D plan document. On searching process, law ontology is used as a knowledge base that have information about legal elements like legal terms, statutes, etc. After getting relevant statutes from the ontology, ranking process based on legal term s importance in both query document and a set of statutes proceeds. Therefore, researchers can earn a list of statutes with ranking which indicates their priority to be checked. Finally, researchers can review relevant statutes and constitute their research plan well. The purpose of this model is to prevent R&D projects from failure in advance, ultimately allowing researchers to search related statute at all times during the R&D project. The rest of this paper is organized as follows: In section 2, we reviewed some related works. We explained our methodology in section 3, and described how we implemented the method as a system in section 4. Finally, we concluded this paper in section 5. 2. Related work In this study, we use ontology as the knowledge base to save statutes. Ontology Engineering is a typical technology that provides a clear way to structure the relationship between objects. Structured ontology can supplement semantic searching by storing the relationship related to the target domain. Ontology is using RDF (Resource Description Framework) and OWL (Web Ontology Language), defined by the standards in the World Wide Web Consortium. RDF and OWL have an advantage in integration with other systems because they are based on XML (extensive Markup Language), a widely used format using tag structure.
There are many cases that utilize ontology to store and represent information. The data from the different sources can be integrated through ontology, and it is possible to interpret semantics of data by using ontology s specified vocabulary. Researches that use ontology in legal domain are as follows. Lame (2005) applied ontology which contains meaningful relationship between the legal concepts to a legal search system, by using NLP (Natural Language Processing) technology. By utilizing text parsing and statistical analysis of the French legal documents, legal concepts are logically structured. Kim (2012) utilized ontologies for users who lack legal knowledge to change general terms into legal terms. Jo (2015) proposed a model based on ontology to build a legal information search engine, but it did not reach the actual implementation. There are not many studies using ontology as a knowledge base for storing relations about statutes. A lot of researches using ontology in legal domain were focused mainly on the relationship between general terms and legal terms. 3. Methodology In this section, we propose a methodology of searching statutes related with a query document and determining their ranking by measures, called relevance scores. The proposed approach is divided into two steps: (1) searching process and (2) ranking process. The overall process can be viewed in Figure 1. Searching process uses relationship between legal terms and statutes defining them. First, terms in query document are extracted using morpheme analyzer. Then only the legal terms are selected using information in law ontology. In the ranking process, we calculate weight of each legal term and determine the ranking of statutes by relevance score, computed using the legal term s weight. The underlying assumption is that importance of a legal term in both a query document and a set of relevant statutes affects relevance of statutes defining it. To calculate the weight, we use TF-IDF, a well-known weighting scheme, to get importance of a legal term in a set of relevant statutes. We also use centrality score computed by page-rank algorithm to get significance of a legal term in query document. We think that a query document can be regarded as a network of legal terms, and use semantic relations between them to get centrality in the document network. Finally, relevance score of a statute is calculated based on weight of legal terms defined by it. In section 3.1, we describe the searching process. We use morpheme analysis to extract nouns from query document and classify legal terms among them. Also, we get a set of statutes defining legal terms by querying law ontology. These statutes are judged to be relevant. In section 3.2, we propose measures to score and rank statutes related with query documents. 3.1 Searching process In our approach, legal terms are important concepts that connect a query document with statutes. To extract legal terms, we first used a morpheme analyzer to select only nouns. We filter out legal terms from the nouns and find a set of statutes defining each of them. These two jobs proceed simultaneously, using information in our developed law ontology. The law ontology is constructed using Open API service provided by Korea Ministry of Government Legislation (http://open.law.go.kr) and contains information of all the legal terms, statutes and relations between them. Ontology schema is shown in Figure 2. Figure 1. Overall process
3.1.1 Law ontology The role of law ontology in law retrieval system is to keep hierarchical relation between statutes and define relation between legal terms and statutes. There are seven classes in law ontology: Statute, Act, Decree, Rule, Domain, LegalTerm, and Organization. In our concerned range, laws are divided into three hierarchical groups: acts, decrees, and rules. Acts are broad principles. Acts need a set of low level principles like decrees and rules, because an act can t define every cases to apply itself. Enforcement decrees are middle-level statutes. They are also known as presidential decrees. Enforcement rules are the last range that we concern in this system. They are enacted by each ministry. There are three properties among them: hasact, hasdecree, and hasrule. Each property has two different domains and one range. A statute can have a set of reference statutes ( hasreference ), and can be grouped by legal domains, like transportation, territory, railroad, etc. ( hasdomain ) Rules are made by ministry, so it has its administering organization ( hasorganization ). 3.1.2 Relation between legal terms and statutes Nouns from query document are classified into two categories: One is legal, and the other is not legal. Nouns in legal group are called legal terms, and they are officially defined by more than one statutes. If a noun is classified as not legal, it is abandoned for the next stage. For example, if there is a noun public transportation in a query document, we search the word in the ontology. Then, a statute named act on the support and promotion of utilization of mass transit system is retrieved as a law defining it. So, public transportation is now categorized as a legal term. One legal term can be defined by many laws, and one law can define multiple legal terms. Therefore, the relation between legal terms and laws is many-to-many. Each noun is queried to the ontology using SPARQL, the standard query language for ontologies, and if it is an individual of class LegalTerm, a set of statutes defining it which are connected by object property isdefinedby is stored for next process. 3.2 Ranking process All the statutes obtained from the previous process have some relevance with query document. But their degree of relevance differs from each other. Therefore, the list of statutes needs to be provided with appropriate ranking, so that researchers can determine which statute should be checked first. Figure 2 Ontology schema In this section, we propose six scoring measures that can evaluate statute s relevance with query document. These measures are calculated based on legal terms importance within both query document and a set of relevant statutes. 3.2.1 Importance of legal term in query document To measure degree of relevance of statutes, we focus on the role of legal terms. If a legal term is important in query document, then statutes defining it have strong possibility of high relevance. We use frequency concept as a basic weighting scheme and compute centrality score to complement it. Centrality score is a measure calculating importance of a node in the network including it. We consider a query document as a network of legal terms. PMI (Pointwise Mutual Information) is used as an edge weight for each node pair. PMI is calculated for a pair of two elements based on the co-occurrence probability. High PMI value means high relevance between them. Unlike original PMI formula, we don t apply logarithm because edge weight can t have value under 0. The network can be represented as a complete graph. After building a network for query document, we apply PageRank algorithm, one of the well-known methods for measuring centrality. PageRank value of a legal term represents relative significance in the document. So the term is near core semantic of the document. 3.2.2 Importance of legal term in a set of relevant statutes We calculate TF IDF to determine importance of legal terms in a set of relevant statutes. TF IDF (Term Frequency Inverse Document Frequency) is a statistical measure that explains how many times a word is used in a document from document set. TF IDF means importance of a legal term in the result set. 3.2.3 Relevance scores We propose six measures called relevance score to rank statutes. The purpose of these measures are to give researchers more information about priority. Concept of our relevance score is as follows. RelevanceScore = (Importance of term in query document) (Importance of term in a set of relevant statutes) That is, the degree of relevance depends on importance of the legal term in query document and a set of statutes. Therefore, a statute which defines core terms, gets high priority.
Three measures are based on only TF IDF concepts, while the others use centrality score to reflect additional information from query document. Sum of contribution (C-SUM): Sum of TF-IDF values of legal terms defined by the statute. We only consider whether it occurs in the whole document. C-SUM is computed as follows: RS C SUM (j) = tf idf(i, j, S), where i is for legal term, j is for statute, ST is a set of legal terms in query document, and S is a set of relevant statutes. Sum of contribution by sections (C-SUM-SC): Sum of TF-IDF values of legal terms defined by the statute, considering the number of sections term appears. We assume that a query document can be divided into multiple sections. So, if a legal term occurs in multiple sections, it can be judged as an important word. RS C SUM SC (j) = tf idf(i, j, k, S) numofsec(i, k), where numofsec is the number of sections that term i appears. Sum of average contribution by sections (C-SUM-AVG-SC): Sum of average contribution in each section. For each section, it adds TF-IDF of legal terms which are defined by the statute, and divides it by the total number of legal terms in each section. RS C SUM AVG SC (j) = tf idf(i, j, k, S), where SC(k) is the number of legal terms in section k. numofsec(i, k) SC(k) These measures are mainly computed by TF-IDF values of legal terms. These are methods of centrality contribution (CN) lines computed by weighted Centrality Score on TF-IDF. These two kind of lines have similar computing process, but different on the point of applying Centrality Score. Sum of network contribution (CN-SUM): C-SUM considering centrality score of the term. RS CN SUM (j) = tf idf(i, j, S) Centrality(i) Sum of network contribution (CN-SUM-SC): C-SUM-SC considering centrality score of the term. RS CN SUM SC (j) = tf idf(i, j, k, S) numofsec(i, k) Centrality(i) Sum of network contribution (CN-SUM-AVG-SC): C-SUM- AVG-SC considering centrality score of the term. RS CN SUM AVG SC (j) = tf idf(i, j, k, S) Centrality(i) numofsec(i, k) SC(k) Finally, rank for each statute based on these relevance scores is assigned. 4. Implementation We implemented a prototype system based on our methodology, and evaluated performance of proposed relevance scores. The purpose of this test is to prove our searching approach is useful in reality and our ranking measures, especially centrality-applied scores, are meaningful. 4.1 Prototype System Target is railroad domain, and summary documents of R&D plan are used as query documents. We used a part of the statute ontology which contains only individuals about railroad. Query documents and knowledge about railroad ontology are provided by KRRI (Korea Railroad Research Institute). To earn PMI values between legal terms, we used number of news articles. The number of articles is obtained by using API from Naver (http://www.naver.com), the most popular portal site in Korea. For example, suppose the number of articles containing both rail and transport is 100, while number of those containing either word is 50 and 30, respectively. Then the simple PMI value for edge weight is 100/50*30 = 1/15. The number of relevant statutes for query document 1 is 78, while document 2 has 73 statutes as result. 4.2 Evaluation of relevance scores To check each relevance score s performance, an answer set for query document is needed to evaluate ranks of retrieved statutes. But, there is two problems. First, an answer set should be based on real relevance of statutes with query document, but the relevance can t be appraised by machines. Second, even R&D experts in railroad domain don t know the whole set of statutes related with query document, which means the answer set is not prepared. To deal with these issues, researchers in KRRI manually evaluate the result of each query document. Namely, an answer set is defined after retrieval. They evaluate relation between query document and retrieved statutes, then assign one of the three degrees among highly relevant, possibly relevant, and irrelevant to each statute. 4.2.1 Evaluation measures To evaluate relevance scores, we use two measures, sum of ranks and average precision-recall. Sum of ranks (SR): Sum of ranks that relevant statutes have. If this value is big, it means lower ranks are assigned to relevant statutes, so performance of the relevance score is not good. Sum of ranks in the best case has a value of k*(k+1)/2 for the number of relevant statutes k. SR is computed as follows: H SR = rank(h) h H, where H is the set of highly relevant statutes and rank(h) is an assigned rank for statute h. If h is not contained in retrieved set, rank(h) is 0. Weighted sum of ranks (WSR): This measure also considers moderate statutes. WSR is computed as follows: H WSR = rank(h) + 0.5 rank(p) h H, where P is the set of possibly relevant statutes. Average precision & average recall (AP & AR): For a size of retrieved statutes set n, we calculate precision and recall for n times and get the average value. Precision of statute k is the number of relevant statutes which has lower rank than the rank of k, divided by the rank of k. P p P
# relevant statutes whose rank < rank(k) Precision(k) = rank(k) And recall of statute k has the same numerator, but the denominator is the number of relevant statutes. Recall(k) = # relevant statutes whose rank < rank(k) # relevant statutes 4.2.2 Evaluation result Sum of ranks and weighted sum of ranks for two summary document are presented in Table 1 and Figure 3. Large value of these measures means low performance. Table 1. Sum of ranks and weighted sum of ranks Document 1 Document 2 SR WSR SR WSR 1. CN-SUM 398 542.5 216 345 2. CN-SUM-SC 413 545 200 331 3. CN-SUM-AVG-SC 415 540.5 212 339.5 4. C-SUM 575 765.5 270 439 5. C-SUM-SC 552 706 230 397.5 6. C-SUM-AVG-SC 546 684.5 253 414.5 Figure 3. Sum of ranks and weighted sum of ranks In Table 2 and Figure 4, average precision and average recall of retrieved statutes are presented. Large value of these measures means high performance. Table 2. Average precision and Average recall Document 1 Document 2 AP AR AP AR 1. CN-SUM 0.55 0.78 0.31 0.767 2. CN-SUM-SC 0.538 0.772 0.357 0.785 3. CN-SUM-AVG-SC 0.533 0.77 0.343 0.771 4. C-SUM 0.465 0.677 0.264 0.705 5. C-SUM-SC 0.464 0.691 0.344 0.751 6. C-SUM-AVG-SC 0.466 0.694 0.343 0.739 Figure 4. Average precision and Average recall We can find out that CN scores show relatively high performance than C scores. That is, relevance scores using centrality score have better performance than those without centrality. 5. CONCLUSION In this research, we propose a statute retrieval system that can find a set of statutes which may affect R&D projects. This system searches statutes that are relevant to the project, considering the contents in query document. We use ontology as a knowledge base and design relevance scores that can help system to assign rank to each statute. This ranking system gives priority to users, helping them to decide which one should be seen before others. Through this system, researchers can check the statutes and adjust direction of their research. Since technologies about ontology are standardized, vocabularies and semantic structure in this research can be easily modified and reused in other ontologies about statutes. In other words, this model is a neutral model regardless of domain. 6. ACKNOWLEDGMENTS This research was supported and funded by Korean Railroad Research Institute. 7. REFERENCES [1] Brin, S. and Page, L., The anatomy of a large-scale hypertextual Web search engine, Computer Networks and ISDN Systems, Vol. 30, pp. 107-117, 1998. [2] Church, K. W. and Hanks, P., Word association norms, mutual information, and lexicography, Comput. Linguist, Vol. 16, No. 1, pp. 22-29, 1990. [3] Jo, D. W., Seo, M. J., and Kim, M. Ho., A Study on Legal Information Retrieval Engine based on Ontology, Korea Information Science Society, pp. 1568-1570, 2015. [4] Kim, H. L., Kim, H. G., Personal Electronic Document Retrieval System Using Semantic Web/Ontology Technologies, The Journal of Society for e-business StudiesVol. 12, No. 1, pp. 135-149, 2007. [5] Kim, J. H., Lee, J. S., Lee, M. J., Kim, W. J., and Hong, J. S., Term Mapping Methodology between Everyday Words and Legal Terms for Law Information Search System, Biblographic Info: J Intell Inform Syst, Vol. 18, No. 3, pp. 137-152, 2012. [6] Lame, G., Using NLP Techniques to Identify Legal Ontology Components : Concepts and Relations, Artificial Intelligence and Law, Vol. 12, No. 4, pp. 169-184, 2005.
[7] Mitschick, A., Ontology-based Indexing and Contextualization of Multimedia Documents for Personal Information Management Applications, International Journal on Advances in Software, Vol. 3, No. 1, 2 pp. 31-40, 2010.