Sorting Networks. Brian Pursley CSC204 5/17/2008

Transcription

1 Sorting Networks Brian Pursley CSC204 5/17/2008 Introduction Sorting is one of the fundamental problems studied in Computer Science. There are numerous algorithms for sorting data, and many more algorithms include sorting as a component. Being able to sort efficiently is important for the efficient design of almost all computer applications. Problem Statement It is known that comparison-based sorting algorithms running on a singleprocessor computer have a lower bound on time complexity of Ω ( nlog n) [3]. This lower bound is due to the algorithm being able to perform only one comparison at a time. Specialized, non-comparison sorting algorithms, such as counting sort and radix sort, can achieve better performance, but are suitable only when the input meets certain requirements necessary in order for those algorithms to work properly [3]. Is it possible to sort n elements faster than Ο ( nlog n) time by comparing elements? Sorting networks offer a way to achieve faster running time through the use of parallel execution. This paper will discuss methods for constructing sorting networks and analyze their efficiency in terms of both cost and depth. Background on Sorting Networks The fundamental building block of a sorting network is the comparator. A comparator is a trivial component that can sort two numbers. It takes in two numbers, a and b, and outputs the minimum of a or b to the top output, and the maximum of a or b to the bottom output by performing a swap if necessary [1]. Figure 1(a) shows a comparator and (b) shows a line representation of a comparator that is often used to illustrate a sorting network s design

2 Comparator Figure 1: (a) A comparator takes two inputs and outputs the min and max. (b) A minimal representation of a comparator that will be used for the remainder of this paper. When there are more than two inputs, multiple comparators can be used together to form a comparison network. Each comparator connects two lines, and any comparators that do not share the same lines can be performed in parallel [4]. Figure 2 shows a simple comparison network where the a 1 /a 2 comparison is done in parallel with the a 3 /a 4 comparison. The a 2 /a 3 comparison must wait for the first two comparisons to finish before the inputs become available. A sorting network is a comparison network that results in a sorted (monotonically increasing) output for all possible inputs. a 1 a 2 a 3 a 4 Figure 2: A comparison network. This comparison network is not a sorting network because the output is not sorted for all possible inputs. There are many ways to construct a sorting network. In Sorting networks and their applications, Batcher describes two methods for constructing sorting networks: Using bitonic sorters, and using the odd-even merge method [1]. A Sorting Network Based on Bitonic Sorters A bitonic sorter is a type of sorting network that accepts a bitonic sequence of numbers as input and generates a sorted sequence of numbers as output. A bitonic sequence is a sequence of numbers that is either ascending, descending, ascending and then descending, or descending and then ascending. It can have, at most, one point within the sequence where the order changes direction [2]. Figure 3 shows some examples of sequences that are bitonic Ascending (monotonically increasing) Descending (monotonically decreasing) Ascending, then descending Descending, then ascending Figure 3: Examples of bitonic sequences - 2 -

3 Because the output of a bitonic sorter is a sorted sequence (monotonic sequence), and a monotonic sequence is itself a bitonic sequence, the output of a bitonic sorter can again be used as the input to another bitonic sorter. This is an important property of the bitonic sorter, allowing small bitonic sorters to be used as building blocks in the construction of larger bitonic sorters [1,2]. A bitonic sorter can be recursively defined as a half-cleaner, plus two bitonic sorters. The half cleaner splits one bitonic sequence into one monotonic (the cleaned half ) and one bitonic sequence, which are then both fed into smaller bitonic sorters. Figure 4 shows an example of an 8-input bitonic sorter than has been constructed recursively using two 4-input bitonic sorters [1,2]. Figure 4: (a) a 4-input bitonic sorter. (b) an 8-input bitonic sorter, showing its recursive construction using a half cleaner and two 4-input bitonic sorters. (c) the complete 8-input bitonic sorter, with all of the comparators shown. In order to construct a bitonic sorting network, we first attempt to use bitonic sorters as building blocks, starting with the smallest 2-input bitonic sorters, which are really just single comparators, and feeding the output of two smaller bitonic sorters into a single larger bitonic sorter, until the entire sequence is sorted [1]. Figure 5(a) shows an example of an 8-input network constructed in this fashion. There is a problem with this design, however, resulting in it failing to be a sorting network. The output of each smaller bitonic sorter is sorted, and two sorted sequences concatenated will not always be bitonic. In order to overcome this problem, the bitonic sorter can be modified to flip (or reverse) the second half of the input sequence. Since both halves of the input sequence are sorted, the input sequence is not bitonic, but the effect of flipping the second half of the input sequence will result in the input sequence becoming a bitonic sequence [3]. This manipulation allows the bitonic sorter to be used as a building block to construct a sorting network of bitonic sorters. An example of a sorting network made of the modified bitonic sorters is shown in Figure 5(b)

4 Figure 5: (a) An incorrect attempt at constructing a bitonic sorting network, because the input to each bitonic sorter will not always be bitonic. This comparison network is not a sorting network. (b) A correct sorting network created from bitonic sorters. The first half cleaner of each bitonic sorter has been modified to flip the second half of the input sequence, so that two sorted halves can be accepted as input instead of a single bitonic sequence being required. A Sorting Network Based on Odd-Even Merge Networks The odd-even merge network is another method of constructing sorting networks introduced by Batcher [1]. An odd-even merge network is a comparison network that can take as input two sorted sequences, and return a single sorted sequence. It does this by sorting the odd inputs, sorting the even inputs, and then combining the results. Figure 6 shows a 2x2 odd-even merge network, and a 4x4 odd-even merge network. Figure 6: (a) A 2x2 odd-even merge network. (b) A 4x4 odd-even merge network. (c) The same 4x4 odd-even merge network, shown with parallel steps combined. Using odd-even merge networks, a sorting network can be designed based on the well-known mergesort algorithm. The algorithm begins with 1x1 odd-even merge networks, which are just single comparators. The output of each pair of 1x1 merge is used as the input of larger 2x2 merge components. This iteration in the design continues into the entire sequence has been sorted [1]. Figure 7 shows a sorting network constructed using odd-even merge networks

5 Figure 7: (a) A block diagram showing the construction of a sorting network using odd-even merge networks. (b) The same sorting network, shown with the comparator detail. Other Sorting Network Designs The bitonic sort and the odd-even merge are two popular designs for sorting networks, but there are many possible ways to construct a sorting network. Other comparison sorting algorithms designed for serial computers, such as insertion sort, bubble sort, and selection sort, can be implemented using a sorting network. Often times, the minimum-cost or minimum-depth sorting networks can be achieved using a sorting network designed for a specific input size, and not following any known pattern [4]. Many specific best known sorting networks have been discovered for specific input sizes, n 16. Some sorting networks are designed to be periodic [8], meaning that in order to complete the sort, the input must be passed multiple times through the same sorting network. An advantage of this approach is that a hardware implementation will require fewer components, since they will be re-used during each pass through the network. Figure 8 shows an example of a comparison network with only 7 comparators that could be used as a periodic sorting network if the output were to be cycled back through as input multiple times. a 1 a 2 a 3 a 4 a 5 a 6 a 7 a 8 Figure 8: An example of a comparison network that could be used as a periodic sorting network. Validating the Correctness of Sorting Networks Sorting networks for large input sizes can become quite complex, making it difficult to validate that a comparison network is indeed a sorting network. In order to check whether a comparison network sorts all possible input sequences would require n! permutations, where n is the number of inputs, across the input sequence to be checked if every possible order of the inputs were to be examined. The actual number of checks required to validate a sorting network is much less than n!, due to the zero-one principle. According to Knuth, the zero-one principle - 5 -

6 says that if a network with n input lines sorts all 2 n sequences of 0s and 1s into nondecreasing order, it will sort any arbitrary sequence of n numbers into nondecreasing order [4]. Analyzing the Complexity of Sorting Networks The complexity of a sorting network is measured in two ways: cost and depth. Cost represents the total number of comparators required to implement the sorting network, while depth represents the maximum number of steps required to sort all inputs. Depth takes into account for the fact that multiple comparators with independent inputs can be executed in parallel [4]. Depending upon the specific application of the sorting network it may be desirable to minimize cost, as in a hardware implementation, or depth, in the case of a parallel implementation. The bitonic sorting network illustrated earlier has a cost of Ο ( nlog 2 n), and the odd-even merge sorting network has a cost of Ο (log 2 n), where n is the number if inputs to the network [6]. Others have found ways to construct sorting networks using Ο( nlog n) comparators [7], although the practical applications of a such a sorting network may be limited [5]. The bitonic sorting network and odd-even merge sorting network have a depth of Ο (log 2 n), where n is the number of inputs to the sorting network. It is possible to construct a sorting network having a depth of Ο (log n) [7], but the constants hidden in the asymptotic notation are prohibitively large for practical applications [5]. Advantages and Limitations of Sorting Networks Because of the parallel nature of a sorting network s design, sorting networks are well suited for implementation on parallel systems, or even an interconnected network of computers. However on single-processor systems, being able to execute multiple comparisons in parallel is not an advantage, and a sorting network may not be as efficient as other algorithms in this case. Sorting networks also translate well into hardware designs, as the logic of a single comparator is simple to manufacture [4]. If the input size is known to be fixed, this is relatively straightforward. But if the input size is arbitrarily large, it becomes more difficult to implement, requiring memory and using a periodic comparison network [6]

7 Possible Future Research There are many opportunities for future research in the areas of sorting networks. Because of increased availability of multiple-core processors in PCs, researchers have focused more on designing algorithms that can take advantage of parallel execution paths. One area of research involving the use of commodity hardware has been in using the specialized processing power found in video card GPUs (Graphics Processing Units) to implement sorting networks for massive amounts of data, with applications in databases and data mining [9]. Another area of future research is in discovering new minimum-cost and minimum-depth sorting networks for specific input sizes. Some minimum-cost and minimum-depth sorting networks have been discovered, but others still remain undiscovered [4]. Still more research has been done in the area of designing fault-tolerant sorting networks, where the failure of individual comparators can be overcome [10]. Conclusion Sorting networks offer a way to sort that that works well in parallel systems and hardware implementations. By running a sorting network on a parallel system, a faster run-time than similar algorithms that are designed for single-processor computers can often be achieved. As hardware improvements continue, sorting networks will certainly play an important role in the future of sorting. References [1] K. E. Batcher, Sorting networks and their applications, Proc. AFIPS Spring Joint Computer Conf., Vol. 32, 1968, pp [2] Kathy J. Liszka, Kenneth E. Batcher, "A Generalized Bitonic Sorting Network," icpp, pp , 1993 International Conference on Parallel Processing - ICPP'93 Vol1, 1993 [3] T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, Introduction to Algorithms, Second Edition, MIT Press, Cambridge, MA/McGraw-Hill, New York, [4] D. E. Knuth, The Art of Computer Programming, vol. 3. Reading, Mass.: Addison-Wesley,

8 [5] M. Ajtai, J. Komlós, and E. Szemerédi, An O(n log n) sorting network, Proceedings of the Fifteenth Annual ACM Symposium on theory of Computing STOC '83. ACM, New York, NY, 1983, pp 1-9. [6] Stephan Olariu, M. Cristina Pinotti, S.Q. Zheng, "How to Sort N Items Using a Sorting Network of Fixed I/O Size," IEEE Transactions on Parallel and Distributed Systems, vol. 10, no. 5, pp , May, 1999 [7] M. Ajtai, J. Komlós, and E. Szemerédi, Sorting in c log n Parallel Steps, Combinatorica 3 (1), 1983, pp [8] M. Dowd, Y. Perl, L. Rudolph, M. Saks, The periodic balanced sorting network, Journal of the ACM (JACM), v.36 n.4, p , Oct [9] N. Govindaraju, J. Gray, R. Kumar, and D. Manocha, "GPUTeraSort: High Performance Graphics Coprocessor Sorting for Large Database Management," ACM SIGMOD, [10] S. Assaf, E. Upfal, "Fault tolerant sorting network," sfcs, pp vol.1, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science,