U.C.

Berkeley CS172: Automata, Computability and Complexity Professor Luca Trevisan

Solutions to Problem Set 1 2/1/2007

Solutions to Problem Set 1

1. Prove that the following languages are regular, either by exhibiting a regular expression representing the language, or a DFA/NFA that recognizes the language: [10 x 3 = 30 points] (a) all strings that do not contain the substring aba, for = {a, b} (for instance, aabaa contains the substring aba, whereas abba does not) Solution: The following machine recognizes the given language by maintaining a state for how much of the string aba it has seen. On seeing aba it goes into a non-accepting state and stays there.
b a b a b a a,b

(7 points for the DFA and 3 for the explanation.) (b) set of strings such that each block of 4 consecutive symbols contains at least two as, for = {a, b} Solution: The following machine remembers the last four characters it has read from the string. The names of the states indicate the (length four) blocks they represent.
a
aaaa

b b
aaab

a b a

aaba

aabb

a a b
abaa abab

b
abba

b a b a
abbb, babb, bbab

a,b

a
baaa

a b
baab

b
baba bbaa

b a

(7 points for the DFA and 3 for the explanation.) (c) set of binary strings ( = {0, 1}) which when interpreted as a number (with the most signicant bit on the left), are divisible by 5. Solution: We maintain the remainder of the number read so far, when divided by 5. To update the remainder, note that if x is the number read so far, and b is the new bit that is read then the new number is y = 2x + b and y mod 5 = ((2x mod 5) + b) mod 5. (6 points for the DFA and 4 for the explanation.)
0 1 0

2
1 0

2. (Sipser, problem 1.31) For any string w = w1 w2 wn , the reverse of w, written as wR is the string w in reverse order, wn w2 w1 . For any language A, let AR = {wR | w A}. Show that if A is regular, so is AR . [20 points] Solution: One solution is recursively (or inductively) dene a reversing operation on regular expressions, and apply that operation on the regular expression for A. In particular, given a regular expression R, reverse(R) is: a for some a , if R = , if R = , (reverse(R1 ) reverse(R2 )), if R = R1 R2 , (reverse(R2 ) reverse(R1 )) if R = R1 R2 , or (reverse(R1 ) ), if R = (R1 ).

(8 points for saying reversing the regular expression, and 12 points for explaining how its done. Its important to point out that the operation is performed recursively.) Another solution is to start with a DFA M for A, and build a NFA M for AR as follows: reverse all the arrows of M , and designate the start state for M as the only accept state qacc for M . Add a new start state q0 for M , and from q0 , add -transitions to each state of M corresponding to accept states of M . It is easy to verify that for any w , there is a path following w from the state start to an accept state in M i there is a path following wR from q0 to qacc in M . It follows that w A i wR AR . (7 points for saying reversing the arrows; 3 points for explaining the new accept state, and 5 points for explaining the new start state and the -transitions. 5 points for explaining, or at least making the nal observation about the paths/connectivity.) 3. We say a string w = w1 w2 . . . wn is a shue of strings u and v if there exists J {1, . . . , n} such that (wj )jJ = u and (wj )j J = v. For example CSS17PR2ING07 is a shue of / the strings CS172 and SPRING07 and in fact, there are two sets J = {1, 2, 4, 5, 8} and J = {1, 3, 4, 5, 8} which work here. We then dene the shue of two languages A and B as S(A, B) = {w| u A, v B s.t. w is a shue of u and v} Show that if A and B are regular languages over a common alphabet , then so is S(A, B). [20 points] Solution: Let MA = (QA , , A , q0A , FA ) and MB = (QB , , B , q0B , FB ) be two DFAs accepting the languages A and B respectively. Then we dene an NFA M = (Q, , , q0 , F ) for S(A, B) as follows. Let Q = QA QB , q0 = (q0A , q0B ) and F = FA FB . Dene ((qA , qB ), s) = {(A (qA , s), qB )} {(qA , B (qB , s))}, i.e., at each step, the machine changes qA according to A or qB according to B . It reaches a state in FA FB if and only if the moves according to A take it from q0A to a state in FA , and the ones according to B take it from q0B to a state in FB . Hence M accepts exactly the language S(A, B). (12 points for designing the machine and 8 for the argument.)

U.C. Berkeley CS172: Automata, Computability and Complexity Professor Luca Trevisan

Solutions to Problem Set 2 2/8/2007

Solutions to Problem Set 2

1. Let k be a positive integer. Let = {0, 1}, and L be the language consisting of all strings over {0, 1} containing a 1 in the kth position from the end (in particular, all strings of length less than k are not in L). [8 + 8 + 14 = 30 points] (a) Construct a DFA with exactly 2k states that recognizes L. Solution: Construct a DFA with one state corresponding to every k-bit string. Formally, let Q = {0, 1}k . We keep track of the last k bits read by the machine. Thus, for a state x1 . . . xk , we dene the transition on reading the bit b as (x1 . . . xk , b) = x2 . . . xk b. Take q0 = 0k and F = {x1 . . . xk Q | x1 = 1}. Note that this does not accept any strings of length less than k (as we start with the all zero state) since the kth position from the end does not exist, and is hence not 1. (b) Construct a NFA with exactly k + 1 states that recognizes L. Solution: We construct an NFA with Q = {0, 1, . . . , k}, with the names of the states corresponding to how many of the last k bits the NFA has seen. Dene (0, 0) = 0, (0, 1) = {0, 1} and (i 1, 0/1) = i for 2 i k. We set q0 = 0 and F = {k}. The machine starts in state 0, on seeing a 1 it may guess that it is the kth bit from the end and proceed to state 1. It then reaches state k and accepts if and only if there are exactly k 1 bits following the one on which it moved from 0 to 1. (c) Prove that any DFA that recognizes L has at least 2k states. Solution: Consider any two dierent k-bit strings x = x1 . . . xk and y = y1 . . . yk and let i be some position such that xi = yi (there must be at least one). Hence, one of the strings contains a 1 in the ith position, while the other contains a 0. Let z = 0i1 . Then z distinguishes x and y as exactly one of xz and yz has the kth bit from the end as 1. Since there are 2k binary strings of length k, which are all mutually distinguishable by the above argument, any DFA for the language must have at least 2k states. 2. [10 + 10 + 10 = 30 points] (a) Let A be the set of strings over {0, 1} that can be written in the form 1k y where y contains at least k 1s, for some k 1. Show that A is a regular language. Solution: It is easy to see that any string in A must start with a 1, and contain at least one other 1 (in the matching y segment). Conversely, any string that starts with a 1 and contains at least one other 1 matches the description for k = 1. Hence, A is described by the regular expression 1 0 1 (0 1) , and is therefore regular. (b) Let B be the set of strings over {0, 1} that can be written in the form 1k 0y where y contains at least k 1s, for some k 1. Show that B is not a regular language. Solution: Assume to the contrary that B is regular. Let p be the pumping length given by the pumping lemma. Consider the string s = 1p 0p 1p B. The pumping lemma guarantees that s can be split into 3 pieces s = abc, where |ab| p. Hence, y = 1i for some i 1. Then, by the pumping lemma, ab2 c = 1p+i 0p 1p B, but cannot be written in the form specied, a contradiction. 1

(c) Let C be the set of strings over {0, 1} that can be written in the form 1k z where z contains at most k 1s, for some k 1. Show that C is not a regular language. Solution: Assume to the contrary that C is regular. Let p be the pumping length given by the pumping lemma. Consider the string s = 1p 0p 1p B. The pumping lemma guarantees that s can be split into 3 pieces s = abz, where |ab| p. Hence, b = 1i for some i 1. Then, by the pumping lemma, ac = 1pi 0p 1p C, but cannot be written in the form specied, a contradiction. 3. Write regular expressions for the following languages: [12 + 8 = 20 points] (a) The set of all binary strings such that every pair of adjacent 0s appears before any pair of adjacent 1s. Solution: Using R(L), to denote the regular expression for the given language L, we must have R(L) = R(L1 )R(L2 ), where L1 is the language of all strings that do not contain any pair of 1s and L2 is the language of all strings that do not contain any pair of 0s. For a string in L1 , every occurrence of a 1, except possibly the last one, must be followed by a 0. Hence, R(L1 ) = (0 + 10) (1 + ). Similarly, R(L2 ) = (1 + 01) (0 + ). Thus, R(L) = (0 + 10) (1 + )(1 + 01) (0 + ), which simplies to (0 + 10) (1 + 01) (0 + ). (b) The set of all binary strings such that the number of 0s in the string is divisible by 5. Solution: Any string in the language must be composed of 0 or more blocks, each having exactly ve 0s and an arbitrary number of 1s between them. This is given by the regular expression (1 01 01 01 01 1 01 ). However, this does not capture the strings containing all 1s, which can be included separately, giving the expression (1 01 01 01 01 01 )+ 1 for the language. 4. We say a string x is a proper prex of a string y, if there exists a non-empty string z such that xz = y. For a language A, we dene the following operation N OEXT EN D(A) = {w A | w is not a proper prex of any string in A} Show that if A is regular, then so is N OEXT EN D(A).[20 points] Solution: Given a DFA for the language A, we want to accept only those strings which reach a nal state, but to which no string can be added to reach a nal state again. Hence, we want to accept strings ending in exactly those nal states, from which there is no (directed) path to any nal state (not even itself). For a given state q F , we can check if there is a path from q to any state in F (or a cycle involving q) by a DFS. Let F F be the set of all the states from which there is no such path. Then changing the set of nal states of the DFA to F gives a DFA for N OEXT EN D(A).

U.C. Berkeley CS172: Automata, Computability and Complexity Professor Luca Trevisan

Solutions to Problem Set 3 2/15/2007

Solutions to Problem Set 3

1. Dene C to be all strings consisting of some positive number of 0s, followed by some string twice, followed again by some positive number of 0. For example 1100 is not in C, since it does not start with at least one 0. However 0001011010000000 is in C since it is three 0s, followed by 101 twice, followed by seven 0s. Prove that C is not regular. [10 points] Solution: We will show that there are innitely many strings, any two of which are distinguishable with respect to C. This will mean there are innitely many indistinguishability classes. By the Myhill-Nerode Theorem, we can then conclude that C is not regular. Our strings will be 01k 0 for each natural number k. Let k1 and k2 be distinct natural numbers. 01k1 01k1 00 is in L. If 01k1 01k2 00 were in L, then it must be 0ss0 or 0ss00 for some string s. So s must contain at least one zero. Thus 01k1 01k2 00 must be 0ss0. So s must end with a 0, and that is the only 0 in s But then must s must be both 1k1 0 and 1k2 0. This is impossible since those strings have dierent lengths. So each 01k 0 is in a dierent indistinguishability class and C is not regular. (4 points for giving the correct strings, 6 points for arguing distinguishability) 2. Let A be the set of all binary strings which, when interpreted as a number with the most signicant bit on the left, are divisible by 5. We know the language is regular from a previous homework. Construct an optimal DFA for A and prove its optimality by giving pairwise distinguishable strings, equal in number to the number of states in your DFA. [10 points] Solution: The following DFA with 5 states recognizes the language (as proved in HW 1). We claim that any two binary strings, which when interpreted as numbers have dierent remainders modulo 5, are distinguishable. This would imply that there must be at least 5 equivalence classes for the indistinguishability relation and hence the DFA here is optimal.
0 1 0

2
1 0

To prove the claim, consider any two strings (thinking of them as numbers) x and y. Let x = r1 mod 5 and y = r2 mod 5, with r1 = r2 . Let w be the number 5 r1 , written using four bits. Then xw (24 x + 5 r1 ) [(24 On the other hand, yw [(24 mod 5)(y mod 5) + 5 r1 ] r2 r1 0 mod 5 mod 5)(x mod 5) + 5 r1 ] 0 mod 5

Hence, the two strings can be distinguished. (3 points for giving the DFA, 2 points for giving the equivalence classes and 5 points for showing the distinguishability.) 1

3. Consider the language F = {ai bj ck | i, j, k 0 and if i = 1 then j = k}. [4 + 4 + 2 = 10 points] (a) Show that F acts like a regular language in the pumping lemma i.e. give a pumping length p and show that F satises the conditions of the lemma for this p. Solution: The pumping lemma says that for any string s in the language, with length greater than the pumping length p, we can write s = xyz with |xy| p, such that xy i z is also in the language for every i 0. For the given language, we can take p = 2. Consider any string ai bj ck in the language. If i = 1 or i > 2, we take x = and y = a. If i = 1, we must have j = k and adding any number of as still preserves the membership in the language. For i > 2, all strings obtained by pumping y as dened above, have two or more as and hence are always in the language. For i = 2, we can take x = and y = aa. Since the strings obtained by pumping in this case always have an even number of as, they are all in the language. Finally, for the case i = 0, we take x = , and y = b if j > 0 and y = c otherwise. Since strings of the form bj ck are always in the language, we satisfy the conditions of the pumping lemma in this case as well. (1 point for handling each of the cases.) (b) Show that F is not regular. Solution: We claim all strings of the form abi must be in distinct equivalence classes for all i 0. This is because any two strings abi1 and abi2 can be distinguished by ci1 , since abi1 ci1 F , while abi2 ci1 F . Since there are innitely many equivalence classes of the indistinguishability relation, we conclude by the Myhill-Nerode theorem that no DFA can recognize F . (2 points for giving a set of distinguishable strings and 2 points for arguing the distinguishability.) (c) Why is this not a contradiction? Solution: The pumping lemma only says that if a language is regular, then it must satisfy the conditions of the lemma. However, this does not necessarily mean that no non-regular language can satisfy these conditions. (2 points) 4. Show that for any positive integer m, there exists a language Am such that: (a) There is a DFA with m states which recognizes Am . (b) No DFA with less than m states recognizes Am . [10 points] Solution 1: Consider the language Am = {1m2 } over the alphabet = {0, 1}. A DFA with m states which has states 0, . . . , m 2 for counting the number of 1s seen so far, and an additional crash state into which it enters on ever seeing a 0 or more than m 2 1s, is a DFA with m states which recognizes this language. On the other hand, any two strings of the form 1i , 1j for 0 i < j m 1 are distinguished by 1m2i and hence, any DFA must have at least m states. Solution 2: Let Am = {1k | m divides k) over = {1}. A DFA with m states which simply stores the number of 1s seen so far, modulo m recognizes this language. Also, for any two 2

strings 1k1 and 1k2 such that k1 k2 mod m, the string 1m(k1 mod m) distinguishes the two. Hence, any two strings in which the number of 1s is dierent modulo m must be in dierent equivalence classes, showing that no DFA with less than m states can recognize this language. (5 points for exhibiting the rst condition and 5 points for the second.)

U.C. Berkeley CS172: Automata, Computability and Complexity Professor Luca Trevisan

Solutions to Problem Set 4 2/23/2007

Solutions to Problem Set 4

1. (Sipser, Problem 3.13) A Turing machine with stay put instead of left is similar to an ordinary Turing machine, but the transition function has the form : Q T Q T {R, S} At each point the machine can move its head right or let it stay in the same position. Show that this Turing machine variant is not equivalent to the usual version. (Hint: Show that these machines only recognize regular languages). [20 points] Solution: It is easy to see that we can simulate any DFA on a Turing machine with stay put instead of left. The only non-trivial modication is to add transitions from state in F to qaccept upon reading a blank, and from states outside F to qreject upon reading a blank. Next, we start with a Turing machine M = (Q, , , , q0 , qaccept , qreject ) with stay put instead of left, and show how we can construct a DFA (Q , , , q0 , F ) that recognizes the same language. The intuition here is that M cannot move left and cannot read anything it has written on the tape as soon as it moves right, and therefore it has essentially only one-way access to its input, much like a DFA. First, we modify M as follows; note that these changes do not aect the language it recognizes. Add a new symbol so that M never writes blanks on the tape; instead, M writes the new symbol when its going to write blanks, and we extend the transition function so that upon reading this new symbol, it behaves as though it read a blank. When M transitions into qreject or qaccept , the reading head moves right (and never stays put). Set Q = Q, = , q0 q, q reject , (q, ) = q , = q0 , and consider the transition function: if q {qaccept , qreject } if M starting at state q and reading keeps staying put. where q is the state the M enters when it rst moves right upon starting at state q and reading .

(for q Q and ). Observe that there are nitely many state-alphabet pairs, M either ends up either staying put and looping, or eventually moves right, and thus is well-dened. Finally, we dene F to be the set containing qaccept and all states q Q, q = qaccept , qreject such that M starting at q and reading blanks, eventually enters qaccept . 2. (Sipser, Problem 3.18) Show that a language is decidable i some enumerator enumerates the language in lexicographic order. [15 points] Solution:If A is decidable by some TM M , the enumerator operates by generating the strings in lexicographic order, testing each in turn for membership in A using M , and printing the string if it is in A. 1

If A is enumerable by some enumerator E in lexicographic order, we consider two cases. If A is nite, it is decidable because all nite languages are decidable (just hardwire each of the strings into the T M ). If A is innite, a TM M that decides A operates as follows. On receiving input w, M runs E to enumerate all strings in A in lexicographic order until some string lexicographically after w appears. This must occur eventually because A is innite. If w has appeared in the enumeration already, then accept; else reject. Note: It is necessary to consider the case where A is nite separately because the enumerator may loop without producing additional output when it is enumerating a nite language. As a result, we end up showing that the language is decidable without using the enumerator for the language to construct a decider. This is a subtle, but essential point. 3. Say that string x is a prex of string y if a string z exists where xz = y, and say that x is a proper prex of y if in addition x = y. A language is prex-free if it doesnt contain a proper prex of any of its members. Let PrexFreeREX = {R|R is a regular expression where L(R) is prex-free} Show that PrexFreeREX is decidable. [15 points] Solution:We construct a TM that decides PrexFreeREX as follows1 . On input R, reject if R is not a valid regular expression. Otherwise, construct a DFA D for the language L(R) (refer to chapter 1 of Sipser for the algorithm that constructs an equivalent NFA for L(R) from R, and for the algorithm that converts an NFA to a DFA). By running a DFS starting from q0 , we can remove all states that are not reachable from q0 from the automaton. Finally, for each accept state q, we run a DFS starting from q and check if another accept state (not equal to q) is reachable from q, or if there is a loop from q to itself. If any such paths or loops are found, reject. Otherwise, accept. Note that it is rst required to remove all the states (actually, just accepting states) not reachable from q0 as these states cannot lead to any string being in the language. 4. Let N on Empty be the following language N on Empty = {< M > | M accepts some string}. Show that N on Empty is Turing recognizable. [10 points] Solution: We simply proceed as in the construction of an enumerator from a Turing machine: simulate M on all strings of length at most i for i steps, and keep increasing i. We accept if the computation of M accepts some string. If L(M ) is non-empty, we are certain that for some i our machine will halt and accept.

Note that PrexFreeREX can contain innite languages. For instance, take R = 0 1.