Recursively Enumerable and Recursive Languages

Definition: A language is recursively enumerable if some Turing machine accepts it

Let
and

L be a recursively enumerable language M the Turing Machine that accepts it

w:
then

For string if

w L
w L

M halts in a final state M halts in a non-final state

or loops forever

if

then

Definition: A language is recursive if some Turing machine accepts it and halts on any input string

In other words: A language is recursive if there is a membership algorithm for it

Let
and

L be a recursive language M the Turing Machine that accepts it

w:
then then

For string if if

w L
w L

M halts in a final state M halts in a non-final state

We will prove: 1. There is a specific language which is not recursively enumerable (not accepted by any Turing Machine)

2. There is a specific language which is recursively enumerable but not recursive

Non Recursively Enumerable

Recursively Enumerable

Recursive

A Language which is not Recursively Enumerable

We want to find a language that is not Recursively Enumerable

This language is not accepted by any Turing Machine

Consider alphabet Strings:

{a}

a, aa, aaa, aaaa,

a a

Consider Turing Machines that accept languages over alphabet

{a}

They are countable:

M1 , M 2 , M 3 , M 4 ,

Example language accepted by M i

L( M i ) {aa, aaaa, aaaaaa}

L( M i ) {a , a , a }
2 4 6

Alternative representation

a
L( M i )

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1

1
0 0

1
0

1
0

L( M 4 )

Consider the language

L {a : a L( M i )}
i i

L consists from the 1s in the diagonal

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1

1
0 0

1
0
3

1
0
4

L( M 4 )

L {a , a ,}

Consider the language

i i

L {a : a L( M i )} L {a : a L( M i )}
i i

consists of the 0s in the diagonal

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1

1
0 0

1
0
1

1
0
2

L( M 4 )

L {a , a ,}

Theorem: Language

is not recursively enumerable

Proof: Assume for contradiction that

is recursively enumerable

There must exist some machine that accepts L

Mk

L( M k ) L

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1

1
0 0

1
0

1
0

L( M 4 )

Question: M k M1 ?

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1
1 1

1
0 0

1
0

1
0

L( M 4 )
Answer:

M k M1

a L( M k ) a L ( M1 )

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1

1
0 0

1
0

1
0

L( M 4 )

Question: M k M 2 ?

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1
2 2

1
0 0

1
0

1
0

L( M 4 )
Answer:

Mk M2

a L( M k ) a L( M 2 )

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1

1
0 0

1
0

1
0

L( M 4 )

Question: M k M 3 ?

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1
3 3

1
0 0

1
0

1
0

L( M 4 )
Answer:

M k M3

a L( M k ) a L( M 3 )

Similarly:

M k Mi

for any

Because either:

a L( M k )
i i

a L( M i )

or

a L( M k )
i i

a L( M i )

Therefore, the machine

M k cannot exist

Therefore, the language L is not recursively enumerable

End of Proof

Observation:
There is no algorithm that describes

(otherwise L would be accepted by some Turing Machine)

Non Recursively Enumerable

L
Recursively Enumerable

Recursive

A Language which is Recursively Enumerable and not Recursive

We want to find a language which

Is recursively enumerable But not recursive

There is a Turing Machine that accepts the language

The machine doesnt halt on some input

We will prove that the language

L {a : a L( M i )}
i i

Is recursively enumerable but not recursive

a
L ( M1 )
L( M 2 )
L( M 3 )

1
0

0 0

1 1 1 1

1
0 0

1
0
3

1
0
4

L( M 4 )

L {a , a ,}

Theorem: The language

L {a : a L( M i )}
i i

is recursively enumerable

Proof: We will give a Turing Machine that accepts L

Turing Machine that accepts For any input string

w
wa
i

Compute i , for which

Find Turing machine

Mi

(using an enumeration procedure for Turing Machines)

Simulate If

M i on input a
End of Proof

M i accepts, then accept w

Observation:
Recursively enumerable i i L {a : a L( M i )}

Not recursively enumerable i i L {a : a L( M i )}

(Thus, also not recursive)

Theorem: The language

L {a : a L( M i )}
i i

is not recursive

Proof: Assume for contradiction that

L is recursive

Then

is recursive:

Take the Turing Machine

M that accepts L

M halts on any input:

If If

M accepts then reject M rejects then accept

Therefore:

L is recursive
But we know:

L is not recursively enumerable

thus, not recursive

CONTRADICTION!!!!

Therefore,

is not recursive

End of Proof

Non Recursively Enumerable

L
Recursively Enumerable

L
Recursive

Turing acceptable languages and Enumeration Procedures

We will prove: If a language is recursive then there is an enumeration procedure for it

(weak result)

A language is recursively enumerable if and only if there is an enumeration procedure for it

(strong result)

Theorem:
if a language L is recursive then there is an enumeration procedure for it

Proof:
Enumeration Machine

~ M

Enumerates all strings of input alphabet

Accepts

If the alphabet is {a, b} then ~ M can enumerate strings as follows:

a b aa ab ba bb aaa aab ......

Enumeration procedure Repeat:

~ generates a string w M M checks if w L

YES: print NO: ignore

w to output w

End of Proof

Example:

L {b, ab, bb, aaa,....}

L( M )
Enumeration Output

~ M

a b aa ab ba bb aaa aab ......

b ab bb aaa

b ab bb aaa

......

......

Theorem:
if language L is recursively enumerable then there is an enumeration procedure for it

Proof:
Enumeration Machine

~ M

Enumerates all strings of input alphabet

Accepts L

If the alphabet is {a, b} then ~ M can enumerate strings as follows:

a b aa ab ba bb aaa aab

NAIVE APPROACH Enumeration procedure Repeat:

~ generates a string w M M checks if w L

YES: print NO: ignore

w to output w

Problem: If w L machine M

may loop forever

BETTER APPROACH

~ Generates first string w M 1

M executes first step on w1

~ Generates second string w M 2

M executes first step on w2

second step on

w1

~ Generates third string w M 3

M executes first step on w3

second step on third step on And so on............

w2 w1

w1
1 Step in string

w2
1

w3
1

w4
1

2 3

2 3

2 3

2 3

If for any string wi machine M halts in a final state then it prints wi on the output

End of Proof

Theorem:
If for language L there is an enumeration procedure then L is recursively enumerable

Proof:

Input Tape

w
Machine that accepts L Enumerator for L

Compare

Turing machine that accepts For input string

Repeat:
Using the enumerator, generate the next string of

L
w

Compare generated string with If same, accept and exit loop

End of Proof

We have proven:

A language is recursively enumerable if and only if there is an enumeration procedure for it

The Chomsky Hierarchy

The Chomsky Hierarchy Non-recursively enumerable

Recursively-enumerable
Recursive Context-sensitive

Context-free
Regular

Decidability

Consider problems with answer YES or NO

Examples: Does Machine Is string Does DFA

M have three states ?

w a binary number?

M accept any input?

A problem is decidable if some Turing machine decides (solves) the problem Decidable problems: Does Machine Is string Does DFA

M have three states ?

w a binary number?

M accept any input?

The Turing machine that decides (solves) a problem answers YES or NO for each instance of the problem

Input problem instance

Turing Machine

YES
NO

The machine that decides (solves) a problem: If the answer is YES then halts in a yes state

If the answer is NO then halts in a no state These states may not be final states

Turing Machine that decides a problem YES states

NO states

YES and NO states are halting states

Difference between Recursive Languages and Decidable problems

For decidable problems:

The YES states may not be final states

Some problems are undecidable:

which means: there is no Turing Machine that solves all instances of the problem

A simple undecidable problem: The membership problem

The Membership Problem

Input: Turing Machine String

Question:

Does M

accept w ?

w L( M ) ?

Theorem: The membership problem is undecidable

(there are M and w for which we cannot decide whether w L ( M ) )

Proof:

Assume for contradiction that the membership problem is decidable

Thus, there exists a Turing Machine that solves the membership problem YES

M H
w

M accepts w M rejects w

NO

Let
Let

L be a recursively enumerable language M be the Turing Machine that accepts L

We will prove that

L is also recursive:

we will describe a Turing machine that accepts L and halts on any input

Turing Machine that accepts and halts on any input

M
w

H M accepts w ?

YES NO

accept w

reject

Therefore,

L is recursive

Since L is chosen arbitrarily, every recursively enumerable language is also recursive But there are recursively enumerable languages which are not recursive Contradiction!!!!

Therefore, the membership problem is undecidable

END OF PROOF

Another famous undecidable problem: The halting problem

The Halting Problem

Input: Turing Machine String

Question:

Does M

halt on input w ?

Theorem: The halting problem is undecidable (there are M and w for which we cannot decide whether M halts on input w )

Proof:

Assume for contradiction that the halting problem is decidable

Thus, there exists Turing Machine that solves the halting problem YES

M H
w

M halts on w M
doesnt halt on

NO

Construction of

H H

Input: initial tape contents

q y YES
q0 qn NO

wM w
Encoding of M

String

Construct machine If If

H :

H returns YES then loop forever H returns NO then halt

H
q y YES

Loop forever

qa

qb

wM w

q0 qn NO

Construct machine
Input:

: H
(machine

wM

M )

If

M halts on input wM
Then loop forever Else halt

wM

copy

wM wM

wM

Run machine
Input:

with input itself: H

(machine

wH

) H

If

halts on input wH H
Then loop forever Else halt

 on input wH H :
If

halts then loops forever H doesnt halt then it halts H

NONSENSE !!!!!

If

Therefore, we have contradiction

The halting problem is undecidable

END OF PROOF

Another proof of the same theorem:

If the halting problem was decidable then every recursively enumerable language would be recursive

Theorem: The halting problem is undecidable

Proof:

Assume for contradiction that the halting problem is decidable

There exists Turing Machine H that solves the halting problem YES

M H
w

M halts on w M
doesnt halt on

NO

Let
Let

L be a recursively enumerable language M be the Turing Machine that accepts L

We will prove that

L is also recursive:

we will describe a Turing machine that accepts L and halts on any input

Turing Machine that accepts and halts on any input

L
w

M
w

H M halts on w ?

NO YES

reject

accept Run M with input w reject

w w

Halts on final state

Halts on non-final state

Therefore

L is recursive

Since L is chosen arbitrarily, every recursively enumerable language is also recursive But there are recursively enumerable languages which are not recursive Contradiction!!!!

Therefore, the halting problem is undecidable

END OF PROOF

Reducibility

Problem

A is reduced to problem B

If we can solve problem we can solve problem A

B then

B
A

Problem

A is reduced to problem B

If

B is decidable then A is decidable

If

A is undecidable then B is undecidable

Example:

the halting problem is reduced to the state-entry problem

The state-entry problem

Inputs: Turing Machine

State
String

q
w

Question: Does

M enter state q on input w ?

Theorem:
The state-entry problem is undecidable

Proof:

Reduce the halting problem to

the state-entry problem

Suppose we have a Decider for the state-entry algorithm:

M
w

state-entry problem decider

YES

M enters q
doesnt enter

NO

We want to build a decider for the halting problem:

M
w

Halting problem decider

YES

M halts on w
doesnt halt on

NO

We want to reduce the halting problem to the state-entry problem:

Halting problem decider

M
w

M q w

State-entry problem NO NO decider

YES

YES

We need to convert one problem instance to the other problem instance

Halting problem decider

M
w

Convert Inputs ?

M q w

State-entry problem NO NO decider

YES

YES

Convert

M to M : q Add new state

From any halting state of

add transitions to

M
halting states

q
Single halt state

M halts on input w
if and only if

M halts on state q on input w

Halting problem decider Generate

M
w

M q w

State-entry problem NO NO decider

YES

YES

We reduced the halting problem to the state-entry problem

Since the halting problem is undecidable, the state-entry problem is undecidable

END OF PROOF

Another example:
the halting problem

is reduced to the blank-tape halting problem

The blank-tape halting problem

Input: Turing Machine

Question: Does

M halt when started with

a blank tape?

Theorem:
The blank-tape halting problem is undecidable

Proof: Reduce the halting problem to the blank-tape halting problem

Suppose we have a decider for the blank-tape halting problem:

blank-tape halting problem NO decider

YES

M halts on
blank tape

M doesnt halt
on blank tape

We want to build a decider for the halting problem:

M
w

halting problem decider

YES

M halts on w
doesnt halt on

NO

We want to reduce the halting problem to the blank-tape halting problem:

Halting problem decider

M
w

Blank-tape M w halting problem NO decider

YES

YES NO

We need to convert one problem instance to the other problem instance

Halting problem decider

M
w

Convert Inputs ?

Blank-tape M w halting problem NO decider

YES

YES NO

Construct a new machine

Mw
w

When started on blank tape, writes Then continues execution like

Mw
step 1 if blank tape then write step2 execute M with input

M halts on input string w

if and only if

M w halts when started with blank tape

Halting problem decider

YES YES blank-tape Generate M w halting problem NO Mw NO decider

M w

We reduced the halting problem to the blank-tape halting problem

Since the halting problem is undecidable, the blank-tape halting problem is undecidable

END OF PROOF

Summary of Undecidable Problems

Halting Problem:

Does machine

M halt on input w ?

Membership problem: Does machine M accept string

w?

Blank-tape halting problem: Does machine M halt when starting on blank tape?

State-entry Problem:

Does machine on input w ?

M enter state q

Uncomputable Functions

Uncomputable Functions

Domain

Values region

A function is uncomputable if it cannot be computed for all of its domain

An uncomputable function:

f ( n)

maximum number of moves until any Turing machine with n states halts when started with the blank tape

Theorem: Function

f (n) is uncomputable

Proof:

Assume for contradiction that f (n) is computable

Then the blank-tape halting problem is decidable

Decider for blank-tape halting problem: Input: machine

M M: m

1. Count states of 2. Compute

f ( m)

3. Simulate M for f ( m) steps starting with empty tape If

M halts then return YES

otherwise return NO

Therefore, the blank-tape halting problem is decidable

However, the blank-tape halting problem is undecidable

Contradiction!!!

Therefore, function

f (n) in uncomputable

END OF PROOF

Undecidable Problems for Recursively Enumerable Languages

Take a recursively enumerable language L

Decision problems: L is empty? L is finite? L contains two different strings of the same length? All these problems are undecidable

Theorem: For any recursively enumerable language it is undecidable to determine whether L is empty

Proof: We will reduce the membership problem to this problem

Let

M be the TM with L( M ) L

Suppose we have a decider for the empty language problem:

empty language problem decider

YES

L( M ) empty L( M ) not empty

NO

We will build the decider for the membership problem:

M
w

membership problem decider

YES

M accepts w M rejects w

NO

We want to reduce the membership problem to the empty language problem:

Membership problem decider

M
w

YES

NO YES

Mw

empty language problem decider

NO

We need to convert one problem instance to the other problem instance

Membership problem decider

M
w

Convert inputs ?

YES

NO YES

Mw

empty language problem decider

NO

Construct machine

Mw :

On arbitrary input string

M w executes the same as with M

When M enters a final state, compare s with w
Accept only if

sw

w L
if and only if

L( M w ) is not empty
L( M w ) {w}

Membership problem decider

M
w

construct M w

Mw

empty language problem decider

YES

NO YES

NO

END OF PROOF

Undecidable problems for Recursively enumerable languages

continued

Take a recursively enumerable language L

Decision problems: L is empty? L is finite? L contains two different strings of the same length? All these problems are undecidable

Theorem: For a recursively enumerable language L it is undecidable to determine whether L is finite

Proof: We will reduce the halting problem to this problem

Let

M be the TM with L( M ) L

Suppose we have a decider for the finite language problem:

finite language problem decider

YES

L( M ) finite L( M ) not finite

NO

We will build a decider for the halting problem:

M
w

YES
Halting problem decider

M halts on w M
doesnt halt on

NO

We want to reduce the halting problem to the finite language problem

Halting problem decider

M
w

finite language Mw problem YES NO decider

YES

NO

We need to convert one problem instance to the other problem instance

Halting problem decider

M
w

convert finite language Mw input problem ? decider

YES

NO YES

NO

Construct machine

M w:

On arbitrary input string

Initially, simulates

M on input w

If M enters a halt state, accept s ( * inifinite language)

Otherwise, reject

( finite language)

M halts on w
if and only if

L( M w ) is infinite
L( M w )
*

halting problem decider

M
w

finite language construct M w problem YES Mw decider NO

YES

NO

Take a recursively enumerable language L

Decision problems: L is empty? L is finite? L contains two different strings of the same length? All these problems are undecidable

Theorem: For a recursively enumerable language L it is undecidable to determine whether L contains two different strings of same length

Proof: We will reduce the halting problem to this problem

Let

M be the TM with L( M ) L

Suppose we have the decider for the two-strings problem:

Two-strings problem decider

YES

L( M ) contains L( M )
Doesnt contain

NO

two equal length strings

We will build a decider for the halting problem:

M
w

YES
Halting problem decider

M halts on w M
doesnt halt on

NO

We want to reduce the halting problem to the empty language problem

Halting problem decider

M
w

M w Two-strings
problem decider

YES

YES NO

NO

We need to convert one problem instance to the other problem instance

Halting problem decider

M
w

convert M Two-strings w problem inputs decider ?

YES

YES NO

NO

Construct machine

Mw :

On arbitrary input string

Initially, simulate

M on input w

When M enters a halt state, accept if s a or s b

(two equal length strings L( M w ) {a, b} )

Otherwise, reject

( L( M w )

M halts on w
if and only if

M w accepts two equal length strings Mw

accepts

and

Halting problem decider

M
w

construct M w Two-strings problem Mw decider

YES

YES NO

NO

Rices Theorem

Definition:
Non-trivial properties of recursively enumerable languages: any property possessed by some (not all) recursively enumerable languages

Some non-trivial properties of recursively enumerable languages: L is empty

L is finite
L contains two different strings of the same length

Rices Theorem: Any non-trivial property of a recursively enumerable language is undecidable

The Post Correspondence Problem

Some undecidable problems for context-free languages:

Is

L(G1 ) L(G2 ) ?
G1, G2 are context-free grammars

Is context-free grammar

G ambiguous?

We need a tool to prove that the previous problems for context-free languages are undecidable:

The Post Correspondence Problem

The Post Correspondence Problem Input:

Two sequences of

n strings

A w1, w2 , , wn B v1, v2 , , vn

There is a Post Correspondence Solution if there is a sequence i, j ,, k such that:

PC-solution:

wi w j wk vi v j vk

Indices may be repeated or omitted

Example:

A:

w1 100 v1 001

w2 11 v2 111

w3 111 v3 11

B:
PC-solution: 2,1,3

w2 w1w3 v2v1v3

11100111

Example:

A:

w1 00 v1 0

w2 001 v2 11

w3 1000 v3 011

B:

There is no solution Because total length of strings from B is smaller than total length of strings from A

We will show: 1. The MPC problem is undecidable

(by reducing the membership to MPC)

2. The PC problem is undecidable

(by reducing MPC to PC)

Theorem: The PC problem is undecidable

Proof: We will reduce the MPC problem to the PC problem

Some undecidable problems for context-free languages:

Is

L(G1 ) L(G2 ) ?
G1, G2 are context-free grammars

Is context-free grammar ambiguous?

We reduce the PC problem to these problems

Theorem: Let G1, G2 be context-free grammars. It is undecidable to determine if L(G1 ) L(G2 )

Proof: Rdeduce the PC problem to this problem

Suppose we have a decider for the empty-intersection problem Context-free grammars

L(G1) L(G2 ) ?
Emptyinterection problem decider YES NO

G1
G2

Theorem: For a context-free grammar

G ,

it is undecidable to determine if G is ambiguous

Proof:

Reduce the PC problem to this problem