Undecidability | CS 139: Theory of Computation

<h1 id="undecidability">Undecidability</h1>

<hr />

<h2 id="administrivia">Administrivia</h2>
<ul>
  <li>Credit/no credit option (deadline is 4/10)</li>
  <li class="fragment">Assignment 6 deadline <strong>extended</strong> to Thursday, 4/2</li>
</ul>

<h1 id="questions">Questions</h1>
<h2 id="about-anything">…about anything?</h2>

<h2 id="high-level-tm-for-a_textdfa">High-Level TM for <code class="language-plaintext highlighter-rouge">$A_\text{DFA}$</code></h2>

<p><code class="language-plaintext highlighter-rouge">$$A_\text{DFA} = \{\langle B,w\rangle \mid B\text{ is a DFA that accepts }w\}$$</code></p>

<hr />

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M = On input &lt;B,w&gt;
  1. Simulate B on w
  2. If B accepts, then ACCEPT
  3. If B rejects, then REJECT
</code></pre></div></div>

<h2 id="high-level-tm-for-all_textdfa">High-Level TM for <code class="language-plaintext highlighter-rouge">$ALL_\text{DFA}$</code></h2>
<p><code class="language-plaintext highlighter-rouge">$$ALL_\text{DFA} = \{\langle A\rangle \mid A\text{ is a DFA and }L(A)=\Sigma^\ast\}$$</code></p>

<hr />

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M = On input &lt;A&gt; where A is a DFA
  1. Mark the starting state of A
  2. Repeat until no more states can be marked this way:
     a. Iterate over each marked state and mark the states that
        are immediately adjacent to it
  3. Check to see if every marked state is an accepting state,
     if so, ACCEPT, if not, REJECT
</code></pre></div></div>

<h2 id="another-example">Another Example</h2>
<p><strong>Claim:</strong> The following language is Turing recognizable:
<code class="language-plaintext highlighter-rouge">$$S = \{\langle M, k\rangle \mid M\text{ is a TM that accepts at least one}\\\text{string $w$ with $|w|\le k$} \}$$</code></p>

<hr />

<ul>
  <li class="fragment"><strong>3-2-1-GO Q:</strong> Do you think the claim is true or false?</li>
</ul>

<h2 id="first-attempt">First-Attempt</h2>

<p><code class="language-plaintext highlighter-rouge">$$S = \{\langle M, k\rangle \mid M\text{ accepts all }w\text{ with $|w|\le k$} \}$$</code></p>

<hr />

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>R = "On input &lt;M,k&gt; where M is a TM and k >= 0,
  1. For each x in {w | w is a string with |w| <= k}
     a. Run M on x
     b. If M accepts, then ACCEPT
  2. REJECT"
</code></pre></div></div>
<ul>
  <li class="fragment">This machine <strong>fails</strong> to recognize <code class="language-plaintext highlighter-rouge">$S$</code>
    <ul>
      <li class="fragment">Consider the input <code class="language-plaintext highlighter-rouge">$\langle M,1\rangle$</code> where <code class="language-plaintext highlighter-rouge">$M$</code> loops forever on <code class="language-plaintext highlighter-rouge">$\epsilon$</code> but accepts <code class="language-plaintext highlighter-rouge">$0$</code></li>
      <li class="fragment">Then <code class="language-plaintext highlighter-rouge">$R$</code> will loop forever when it should accept</li>
    </ul>
  </li>
</ul>

<h2 id="correct-proof">Correct Proof</h2>
<ul>
  <li>We prove the claim true by constructing a machine <code class="language-plaintext highlighter-rouge">$R$</code> that recognizes <code class="language-plaintext highlighter-rouge">$S$</code></li>
  <li class="fragment">Consider the Turing machine <code class="language-plaintext highlighter-rouge">$R$</code> defined by:</li>
</ul>

<div class="language-plaintext fragment highlighter-rouge"><div class="highlight"><pre class="highlight"><code>R = "On input &lt;M,k&gt; where M is a TM and k >= 0,
  1. For each n = 1, 2, 3, 4, ...
     a. For each x in {w | w is a string with |w| <= k}
     b. Run M on x for n steps
     c. If M accepts, then ACCEPT
</code></pre></div></div>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>R = "On input &lt;M,k&gt; where M is a TM and k >= 0,
  1. For each n = 1, 2, 3, 4, ...
     a. For each x in {w | w is a string with |w| <= k}
       i. Run M on x for n steps
       ii. If M accepts, then ACCEPT
</code></pre></div></div>

<ul>
  <li>To see that <code class="language-plaintext highlighter-rouge">$R$</code> recognizes <code class="language-plaintext highlighter-rouge">$S$</code>, consider <code class="language-plaintext highlighter-rouge">$\langle M, k\rangle \in S$</code>
    <ul>
      <li class="fragment">Then <code class="language-plaintext highlighter-rouge">$M$</code> must accept some short string <code class="language-plaintext highlighter-rouge">$w$</code> in a finite number of steps, say <code class="language-plaintext highlighter-rouge">$n$</code> total steps</li>
      <li class="fragment">Our machine <code class="language-plaintext highlighter-rouge">$R$</code> will eventually run <code class="language-plaintext highlighter-rouge">$M$</code> on all short strings for <code class="language-plaintext highlighter-rouge">$n$</code> steps after <code class="language-plaintext highlighter-rouge">$n$</code> iterations of the outer loop—thus it will ACCEPT</li>
    </ul>
  </li>
  <li class="fragment">If <code class="language-plaintext highlighter-rouge">$R$</code> is given a string <code class="language-plaintext highlighter-rouge">$\langle M, k\rangle\not\in S$</code>, then it will loop forever and never accept</li>
  <li class="fragment">Thus <code class="language-plaintext highlighter-rouge">$R$</code> recognizes only the strings in <code class="language-plaintext highlighter-rouge">$S$</code></li>
</ul>

<h2 id="another-example-1">Another Example</h2>
<ul>
  <li><strong>Claim:</strong> The set of Turing recognizable languages is closed under concatenation.</li>
</ul>

<hr />

<ul>
  <li class="fragment"><strong>3-2-1-GO Q:</strong> What does it mean for a set to be <strong>closed</strong> under concatenation?</li>
  <li class="fragment"><strong>3-2-1-GO Q:</strong> Is this claim true?</li>
</ul>

<h2 id="proof-the-claim-is-true">Proof the Claim is True</h2>
<ul>
  <li>We prove the claim true</li>
  <li class="fragment">Let <code class="language-plaintext highlighter-rouge">$A$</code> and <code class="language-plaintext highlighter-rouge">$B$</code> be Turing recognizable languages</li>
  <li class="fragment">It now suffices to show that <code class="language-plaintext highlighter-rouge">$A\circ B$</code> is also Turing recognizable</li>
  <li class="fragment">Let <code class="language-plaintext highlighter-rouge">$M_A$</code> and <code class="language-plaintext highlighter-rouge">$M_B$</code> be the Turing machines that recognize <code class="language-plaintext highlighter-rouge">$A$</code> and <code class="language-plaintext highlighter-rouge">$B$</code>, respectively</li>
  <li class="fragment">Let <code class="language-plaintext highlighter-rouge">$M_{A\circ B}$</code> be defined as:</li>
</ul>

<div class="language-plaintext fragment highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M_AB = On input w,
  1. Nondeterministically choose a number, k, between 0...|w|
  2. Let x:=w[:k] and y:=w[k:]   (note that w = xy)
  3. Simulate M_A on x and M_B on y in parallel---alternating steps
  4. If both machines accept, then ACCEPT; otherwise REJECT
</code></pre></div></div>

<ul>
  <li>To see why <code class="language-plaintext highlighter-rouge">$M_{A\circ B}$</code> recognizes <code class="language-plaintext highlighter-rouge">$A\circ B$</code>, consider a string <code class="language-plaintext highlighter-rouge">$w\in A\circ B$</code>
    <ul>
      <li class="fragment">Then there is a way to cut <code class="language-plaintext highlighter-rouge">$w$</code> into two parts <code class="language-plaintext highlighter-rouge">$w = xy$</code> such that <code class="language-plaintext highlighter-rouge">$x\in A$</code> and <code class="language-plaintext highlighter-rouge">$y\in B$</code></li>
      <li class="fragment">Since <code class="language-plaintext highlighter-rouge">$M_{A\circ B}$</code> nondeterministically selects all possible cuts of <code class="language-plaintext highlighter-rouge">$w$</code>, one of them is <code class="language-plaintext highlighter-rouge">$xy$</code></li>
      <li class="fragment">Since <code class="language-plaintext highlighter-rouge">$M_A, M_B$</code> recognize <code class="language-plaintext highlighter-rouge">$A, B$</code>, they will both halt and accept <code class="language-plaintext highlighter-rouge">$x, y$</code>, respectively</li>
      <li class="fragment">Thus <code class="language-plaintext highlighter-rouge">$M_{A\circ B}$</code> will halt and accept</li>
    </ul>
  </li>
</ul>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M_AB = On input w,
  1. Nondeterministically choose a number, k, between 0...|w|
  2. Let x:=w[:k] and y:=w[k:]   (note that w = xy)
  3. Simulate M_A on x and M_B on y in parallel---alternating steps
  4. If both machines accept, then ACCEPT; otherwise REJECT
</code></pre></div></div>
<ul>
  <li>If <code class="language-plaintext highlighter-rouge">$M_{A\circ B}$</code> is given <code class="language-plaintext highlighter-rouge">$w\not\in A\circ B$</code>, then there is not way to cut it up such that one part is in <code class="language-plaintext highlighter-rouge">$A$</code> and one is in <code class="language-plaintext highlighter-rouge">$B$</code></li>
  <li class="fragment">Thus all nondeterministic branches will either (a) halt and reject or (b) get into an infinite loop</li>
  <li class="fragment">Thus <code class="language-plaintext highlighter-rouge">$M_{A\circ B}$</code> will not accept <code class="language-plaintext highlighter-rouge">$w$</code></li>
  <li class="fragment">Thus it will accept if and only if <code class="language-plaintext highlighter-rouge">$w\in A\circ B$</code>—so it is a recognizer for it</li>
</ul>

<h1 id="undecidability-1">Undecidability</h1>

<h2 id="review-of-last-time">Review of Last Time</h2>
<ul>
  <li>The set of languages <code class="language-plaintext highlighter-rouge">$\mathbb{B} = \{L\mid L\subseteq\{0,1\}^\ast\}$</code> is <strong>uncountable</strong></li>
  <li class="fragment">The set of Turing machines <code class="language-plaintext highlighter-rouge">$\{\langle M\rangle \mid M\text{ is a TM}\}$</code> is <strong>countable</strong></li>
  <li class="fragment">So most languages are <strong>undecidable</strong></li>
</ul>

<h2 id="review-of-last-time-1">Review of Last Time</h2>
<p><strong>Theorem:</strong> The following language is Turing recognizable:
<code class="language-plaintext highlighter-rouge">$$\text{HALT} = \{\langle M, w\rangle \mid M\text{ is a TM that halts on } w\}$$</code></p>

<hr />

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>U = On input &lt;M,w&gt; where M is a Turing machine
  1. Simulate M on w
  2. If M halts, then ACCEPT
</code></pre></div></div>

<hr />

<ul>
  <li class="fragment">Although this demonstrates that <code class="language-plaintext highlighter-rouge">$\text{HALT}$</code> is recognizable, last time we proved that this language is <strong>undecidable</strong></li>
</ul>

<h2 id="proof-idea-by-contradiction">Proof Idea: (by contradiction)</h2>
<ul>
  <li>Assume that <code class="language-plaintext highlighter-rouge">$\text{HALT}$</code> is decidable</li>
  <li class="fragment">Then there exists a Turing machine <code class="language-plaintext highlighter-rouge">$K$</code> that decides it</li>
  <li class="fragment">To reach a contradiction, we constructed a TM that causes <code class="language-plaintext highlighter-rouge">$K$</code> to output the wrong answer</li>
  <li class="fragment">This machine was carefully constructed using Cantor’s <strong>diagonalization</strong> method to make sure that <code class="language-plaintext highlighter-rouge">$K$</code> will fail if given input <code class="language-plaintext highlighter-rouge">$\langle M_n, s_n\rangle$</code> for some <code class="language-plaintext highlighter-rouge">$n\in\mathbb{N}$</code></li>
</ul>

<h2 id="undecidability-2">Undecidability</h2>
<ul>
  <li>Proving that <code class="language-plaintext highlighter-rouge">$\text{HALT}$</code> was undecidable was <strong>a lot of work</strong>!</li>
  <li class="fragment">Many other languages are also undecidable, and it would be useful if there was an <strong>easier way</strong> to prove a language is undecidable</li>
  <li class="fragment">Luckily there is a way!</li>
</ul>

<h1 id="reductions">Reductions</h1>

<h2 id="reductions-1">Reductions</h2>
<ul>
  <li>As programmers, you use reductions <strong>all the time</strong> and don’t even realize it</li>
  <li class="fragment">For example, suppose I ask you to implement the following Python function named <code class="language-plaintext highlighter-rouge">is_even</code>:</li>
</ul>

<div class="language-py fragment highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">is_even</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
    <span class="s">"""Checks whether the integer n is even"""</span>
    <span class="k">return</span> <span class="n">____________________________</span>
</code></pre></div></div>

<ul>
  <li class="fragment"><strong>3-2-1-GO Question:</strong> What goes in the blank in order to complete the function definition?
    <ul>
      <li class="fragment"><code class="language-plaintext highlighter-rouge">n % 2 == 0</code></li>
    </ul>
  </li>
</ul>

<h2 id="reductions-2">Reductions</h2>
<div class="language-py highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">is_even</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
    <span class="s">"""Checks whether the integer n is even"""</span>
    <span class="k">return</span> <span class="n">n</span> <span class="o">%</span> <span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span>
</code></pre></div></div>

<ul>
  <li>Here we <strong>reduced</strong> an instance of one problem to another. In particular…
    <ol>
      <li class="fragment">Checking if <code class="language-plaintext highlighter-rouge">n</code> is even</li>
      <li class="fragment">Checking if <code class="language-plaintext highlighter-rouge">n % d == r</code></li>
    </ol>
  </li>
</ul>

<h2 id="reductions-3">Reductions</h2>
<ul>
  <li>We can even formalize this reduction using languages and Turing machines
    <ul>
      <li class="fragment"><code class="language-plaintext highlighter-rouge">$A = \{\langle n\rangle \mid n\in\mathbb{Z}\text{ is even}\}$</code></li>
      <li class="fragment"><code class="language-plaintext highlighter-rouge">$B = \{\langle n, d, r\rangle \mid n//d\text{ has remainder }r\}$</code></li>
    </ul>
  </li>
  <li class="fragment">Given a Turing machine <code class="language-plaintext highlighter-rouge">$M_B$</code> that decides <code class="language-plaintext highlighter-rouge">$B$</code>, we can now implement a Turing machine that decides <code class="language-plaintext highlighter-rouge">$A$</code>:</li>
</ul>

<div class="language-plaintext fragment highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M_A = "On input &lt;n&gt;, an integer,
  1. Run M_B on &lt;n,2,0&gt;
  2. If M_B accepts, then ACCEPT
  3. If M_B rejects, then REJECT"
</code></pre></div></div>

<h2 id="reductions-4">Reductions</h2>
<ul>
  <li>Since we solved an instance of <code class="language-plaintext highlighter-rouge">$A$</code> by converting it to an instance of <code class="language-plaintext highlighter-rouge">$B$</code>, we say that we <code class="language-plaintext highlighter-rouge">$A$</code> <strong>is reducible to</strong> <code class="language-plaintext highlighter-rouge">$B$</code></li>
  <li class="fragment">From this reduction, we can deduce a few things:
    <ol>
      <li class="fragment">If <code class="language-plaintext highlighter-rouge">$B$</code> is decidable, then <code class="language-plaintext highlighter-rouge">$A$</code> is also decidable</li>
      <li class="fragment">If <code class="language-plaintext highlighter-rouge">$B$</code> is Turing recognizable, then <code class="language-plaintext highlighter-rouge">$A$</code> is also Turing recognizable</li>
    </ol>
  </li>
</ul>

<hr />

<ul>
  <li class="fragment"><strong>3-2-1-GO Q:</strong> What is the contrapositive of 1?
    <ol>
      <li class="fragment">If <code class="language-plaintext highlighter-rouge">$A$</code> is undecidable, then <code class="language-plaintext highlighter-rouge">$B$</code> is also undecidable</li>
    </ol>
  </li>
  <li class="fragment">This will be our primary tool to prove that languages are <strong>undecidable</strong></li>
</ul>

<h2 id="another-undecidability-proof">Another Undecidability Proof</h2>
<p><strong>Claim:</strong> The following language is undecidable:
<code class="language-plaintext highlighter-rouge">$$\text{TOTAL} = \{\langle M\rangle \mid M\text{ is a TM that halts on all inputs}\}$$</code></p>

<hr />

<ul>
  <li><strong>Proof idea:</strong> Reduce a language we already know is undecidable to <code class="language-plaintext highlighter-rouge">$\text{TOTAL}$</code>
    <ul>
      <li class="fragment">A few good candidates: <code class="language-plaintext highlighter-rouge">$A_\text{TM}$</code> and <code class="language-plaintext highlighter-rouge">$\text{HALT}$</code></li>
    </ul>
  </li>
</ul>

<h2 id="a-brief-review">A Brief Review</h2>
<ul>
  <li>Remember when we reduced <code class="language-plaintext highlighter-rouge">$A$</code> to <code class="language-plaintext highlighter-rouge">$B$</code>? We used the following:</li>
</ul>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M_A = "On input &lt;n&gt;, an integer,
  1. Run M_B on &lt;n,2,0&gt;
  2. If M_B accepts, then ACCEPT
  3. If M_B rejects, then REJECT"
</code></pre></div></div>

<ul>
  <li class="fragment">Whenever we reduce a language to another, it means taking a string <code class="language-plaintext highlighter-rouge">$w \in A$</code> and converting it to a string <code class="language-plaintext highlighter-rouge">$\widehat{w}\in B$</code> that will help us solve the problem to <code class="language-plaintext highlighter-rouge">$A$</code></li>
</ul>

<h2 id="another-undecidability-proof-1">Another Undecidability Proof</h2>
<ul>
  <li>To reduce an undecidable problem like <code class="language-plaintext highlighter-rouge">$\text{HALT}$</code> to <code class="language-plaintext highlighter-rouge">$\text{TOTAL}$</code> we need to convert an instance of <code class="language-plaintext highlighter-rouge">$\text{HALT}$</code> such as <code class="language-plaintext highlighter-rouge">$\langle M,w\rangle$</code> into an instance <code class="language-plaintext highlighter-rouge">$\langle \widehat{M}\rangle$</code> of <code class="language-plaintext highlighter-rouge">$\text{TOTAL}$</code></li>
  <li class="fragment">Let’s start the proof!</li>
</ul>

<h2 id="proof-by-contradiction">Proof (by contradiction)</h2>
<ul>
  <li>Assume that <code class="language-plaintext highlighter-rouge">$\text{TOTAL}$</code> is decidable and so there is a Turing machine <code class="language-plaintext highlighter-rouge">$R$</code> that decides it</li>
  <li class="fragment">It now suffices to construct a TM that decides <code class="language-plaintext highlighter-rouge">$\text{HALT}$</code></li>
  <li class="fragment">Let <code class="language-plaintext highlighter-rouge">$S$</code> be the Turing machine:</li>
</ul>

<div class="language-plaintext fragment highlighter-rouge"><div class="highlight"><pre class="highlight"><code>S = "On input &lt;M,w&gt; where M is a Turing machine,
  1. Construct a Turing machine named M* from M and w
  2. Run the machine R on &lt;M*&gt;
  3. If R accepts, then ACCEPT; otherwise REJECT"
</code></pre></div></div>

<div class="language-plaintext fragment highlighter-rouge"><div class="highlight"><pre class="highlight"><code>M* = "On input x,
  1. If x != w, then ACCEPT
  2. If x == w, then run M on w
  3. If M halts, then ACCEPT; otherwise enter an infinite loop"
</code></pre></div></div>

<ul>
  <li class="fragment">If <code class="language-plaintext highlighter-rouge">$M$</code> halts on <code class="language-plaintext highlighter-rouge">$w$</code>, then <code class="language-plaintext highlighter-rouge">$M^\ast$</code> is designed to halt on all inputs. If <code class="language-plaintext highlighter-rouge">$M$</code> loops forever on <code class="language-plaintext highlighter-rouge">$w$</code>, then <code class="language-plaintext highlighter-rouge">$M^\ast$</code> will also loop forever on at least one input</li>
</ul>

<h2 id="another-example-2">Another Example</h2>
<ul>
  <li>Show that the following language is undecidable
 <code class="language-plaintext highlighter-rouge">$$T = \{\langle M \rangle \mid M\text{ is a TM that accepts }w^R\\
 \text{whenever it accepts }w\}$$</code></li>
</ul>