算法DataStructure2_LectureNotes_w7

Last Section•ftp://211.71.1.204•download-ben •Password: studentProbabilistic Algorithm •NP –Completeness•Exact algorithm vs. Heuristics •Deterministic vs. Probabilistic •Approximation algorithmApproximation Algorithm•Bisection method•Position method•Newton’s method•Difference Bound•Performance ratio•TSP (Nearest-neighbor algorithm)Randomized Algorithms •Finding a Good Student •Randomized Quicksort•Testing String Equality •Verification of Matrix ProductsVerification of Matrix ProductsInput: Three n×n matrices,A, B and C.Question: Does A×B = C?Verification of Matrix Products Deterministic algorithms：Trivial Algorithms: O(n3)Strassen: O(n2.81)Best deterministic algorithm: O(n2.376)Verification of Matrix Products Freivalds' AlgorithmAB=C=•Compute the sum of a random subset of the rows of both AB and C.•If the sums are different, return false (AB ≠C).•Otherwise, either return true or try again to improve confidence.⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡−−202112201⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡−−201112201Verification of Matrix Products •The sum of any set of rows of a matrix may be computed by multiplying on the left by a row vector of which each element is either 0 or 1, e.g[1 0 1]×= [1 0 2]+[2 0 2]⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡−−202112201Verification of Matrix Products •Note that S×(A×B)= (S×A)×B. To compute (S×A)×B, where S is a row vector (i.e. 1×n matrix):Y = S×A requires O(n2) time; andY×B requires O(n2) time (because Y is 1×n).•We may therefore compute the sum of any subset of rows of A×B in O(n2) time.Verification of Matrix Products •Now consider the following O(n2) time probabilistic algorithm (Monte-Carlo)Repeat k times {Generate a random row vector S from {0, 1};If (SA)B ≠SC, then return AB≠C;}Return AB=C.Complexity analysis: O(n2)Verification of Matrix Productsz Suppose there is an error on row i of C, i.e. row i of C does not equal row i of AB.z We make no assumptions about the other rows of C.z Also, assume that we select any subset of the n rows with equal probability.Verification of Matrix Productsz There are 2n subsets of the n rows. For example, if n=3, then we have 8 subsets, i.e. (1, 2, 3), (1, 2), (1, 3), (2, 3), (1), (2), (3), and the empty set.z We form 2n-1pairs of subsets so that in each pair, the subsets are the same except that one subset contains row i and the other does not. For example, if n=3 and i=2, then we have 4 pairs, i.e. {(1, 2, 3), (1, 3)}, {(1, 2), (1)}, {(2, 3), (3)} and {(2), the empty set}Verification of Matrix Productsz Consider any pair (S, S’), where S contains i, but S’does not.z Suppose we represent S and S’by row vectors. Then (SAB −SC) −(S’AB −S’C)= S(AB −C) −S’(AB −C)= (S −S’)(AB −C)≠0z Hence, at most one of the following is true:SAB = SC or S’AB = S’CVerification of Matrix Productsz For at most 2n-1of the 2n subsets, the sums of the selected rows of AB and C will be equal.z If C ≠AB, then a single trial will fail to detect an error with probability at most 1/2.z If k trials are done, the algorithm give a wrong result with probability at most 1/2k.z For example, if k=10, then the error probability is at most 0.1%.Pseudo Random Number Generator •计算机是一种精确的，并能准确做出判断的机器，要用来产生完全随机的数列，是自相矛盾的。

•故需要采用一些特殊的方法，生成一些满足独立性，均匀性，随机性，周期长，且计算速度快，占用内存小的一些伪随机序列。

Linear CongruentialGenerator •线性同余生成器利用求余运算，来获得一个伪随机序列:x 0=d (seed)a,b,d 为正整数，m 应充分大))(mod (1m b ax x i i +=−Multiplicative Congruential Generator z Most random number generators generate a sequence of integers by the following recurrence:X 0= a given integer (seed), X i +1= a X i (mod M)For example, for X 0=1, a =5, M=13, we haveX 1=5 mod 13 = 5, X 2= 25 mod 13 = 12,X 3= 60 mod 13 = 8.Each integer in the sequence lies in the range [0, M -1].z Random numbers r i in the range [0,1] is obtained byr i = X i /MMultiplicative Congruential Generator z The most famous generator （Park & Miller）haveM=231-1 = 2147483647, a= 16807.z Schrage solved the difficulty of overflow by using a factorization (因数分解) of M, i.e.M = aq+ r, i.e., q= ⎣M/a⎦and r= M mod a = M –aq. Thena X mod M=a×(X mod q) -r×⎣X/ q⎦if ≥0= a×(X mod q) -r×⎣X/ q⎦+M otherwiseMultiplicative Congruential GeneratorFor the most famous generator, we haveq= ⎣2147483647/16807⎦= 127773 and r= 2836. Note:A random number between m and n(m≤n)can be obtained byX=m+(n-m) ×r,where r is a random number between 0 and 1.Lower Bound ArgumentsLower Boundsz Definition: A lower bound of a problem is the least amount of computation that any algorithm must use to solve this problem.z Lower bounds are generally theoretically derived. z The lower bound for a problem is not unique,but the higher the better.9For example, Ω(1), Ω(n) and Ω(n log n) are all lower bounds for sorting but among them Ω(n log n) is the best.Lower BoundsAt present, if the best lower bound of a problem is Ω(n) and the time complexity of the best algorithm is O(n2). This means that9we may try to find a better lower bound, e.g. Ω(n log n);9we may try to find a better algorithm, e.g. O(n log n);9both of the lower bound and the algorithm may be improved.z If the present lower bound is Ω(n log n) and there is an algorithm with time complexity O(n log n), then the algorithm is optimal.Lower Bound Arguments •Trivial Lower Bounds•Information-Theoretic Arguments •Adversary Arguments•Problem ReductionLower Bound on Sortingz Comparison sort: A family of algorithms that use comparisons to determine the sorted order of a list.z Examples: Mergesort, Quicksortz Decision tree = A tree that describes the comparisons required to sort a list of data.Decision Treesz Given n items there are n! different ways to arrange them (permutation (排列)).z Only one of the permutations is in sorted order. z The aim of sorting is to discover the sorted permutation.z The worst case running time corresponds to the longest path from the root to a leaf.The power of comparisonz Given n, we have M = n! permutations.z Each permutation must appear as one of the leaves in the decision tree.z Every time we perform a comparison, one of two branches eliminates (排除) at most half of the permutations.The power of comparisonz Thus, if sorting is done along the longest path,M comparisons.we will use at least k= log2z Another proof is that since any binary tree of height k has at most 2k leaves, we have 2k≥n!. z The best general sorting algorithm requires (n!) comparisons in the worst case.log2How much is log 2(n !)?Method 1z log 2(n !) = log 2(n (n −1) (n −2) …(2) (1))= log 2n +log 2(n −1)+log 2(n −2)+ …+log2(2)+log 2(1)≥log 2n +log 2(n −1)+log 2(n −2)+ …+log2(n /2)≥n /2 log 2(n /2) = n /2 (log2n –1) = Ω(n log n )Comparison-based Sorting Theorem 1. Any comparison-based sorting algorithm must use Ω(n log n) comparisons for sorting an array of n elements in the worst case.z This means that it is impossible to perform sorting using O(n) or O(log n) or O(n log(log n)) comparisons.Determining The Maximum Theorem 2. Every comparison-based algorithm for determining the maximum of a set of n elements must use at least n/2 comparisons.•Proof. Every element must participate in at least one comparison; if not, the uncompared element can be chosen to be the maximum. Each comparison compares 2 elements. Hence, at least n/2 comparisons must be made.Determining The Maximum Theorem 3.Every comparison-based algorithm for determining the maximum of a set of n elements must use at least n−1 comparisons.•Proof.To say that a given element, x, is the maximum, implies that every other element has lost at least one comparison with another element (not necessarily x). Each comparison produces at most one loser. Hence at least n−1 comparisons must be used.Lower Bound for Finding Max and Min •Remember: we can compute both the maximum and minimum of a list of n numbers, using (3/2)n−2 comparisons. We will prove that this solution is optimal.•Here is a simpler algorithm: Compare x[1] with x[2], x[3] with x[4], etc. We get n/2 maxima and n/2 minima. Find the maximum on the n/2 maxima with n/2−1 comparisons, and the same for the minima. The total cost: n/2+n/2−1+n/2−1 = 3n/2−2 (when n even).Lower Bound for Finding Max and Min Note that x is the minimum and y is the maximum only ifz every element other than x has won at least one comparison andz every element other than y has lost at least one comparison.Lower Bound for Finding Max and Min •Call a win “W”, a loss “L”. For each element, we labelit according to the following 4 cases:z N: never compared with anybody;z W: won all comparisons;z L: lost all comparisons;z WL: won at least one and lost at least one.•The algorithm must assign n−1 W's and n−1 L's. i.e., it MUST need 2n−2 “units of information”.•More precisely, n-2 WL's, 1 W and 1 L.Lower Bound for Finding Max and Min z To prove that the worst case can happen, we will use what is called an adversary (反对者) argument (论证、论据). An adversary is an opponent (对手) that the comparison algorithm plays against.z Its ultimate (最终) goal is to maximize the number of comparisons that the algorithm makes (minimize the amount of information obtained) by constructing an input to the problem.Lower Bound for Finding Max and Min z The adversary tentatively (暂时地) assigns values tothe elements, which may change over time.z However, they can only change in a fashion CONSISTENT (一致的) with previous answers.z That is, an element labeled L may only DECREASE in value (since it has lost all previous comparisons, if it is decreased, it will still lose all of them), and an element labeled W may only INCREASE in value.An element labeled WL cannot change.Lower Bound for Finding Max and Min Theorem 4. Every comparison-base method for determining both the maximum and minimum of a set of n numbers must use at least (3n/2)−2 comparisons, in the worst case.•Proof.Assume n is even. As we have seen above, the algorithm must learn 2n−2 units of information. The most it can learn in one comparison is 2 units; but this can only happen in the case (N,N), in which both elements have never participated before in a comparison. This can only happen at most n/2 times. To learn the remaining n−2 units of information, the algorithm must perform n−2 comparisons in the worst case. The total number of comparisons is therefore at least n/2 + n−2 = (3/2)n−2.End of Section。