Theory of Knowing Machines: Revisiting Gödel and the Mechanistic Thesis

. Church-Turing Thesis, mechanistic project, and G¨odelian Arguments oﬀer diﬀerent perspectives of informal intuitions behind the relationship existing between the notion of intuitively provable and the deﬁnition of decidability by some Turing machine. One of the most formal lines of research in this setting is represented by the theory of knowing machines, based on an extension of Peano Arithmetic, encompassing an epistemic notion of knowledge formalized through a modal operator denoting intuitive provability . In this framework, variants of the Church-Turing Thesis can be constructed and interpreted to characterize the knowledge that can be acquired by machines. In this paper, we survey such a theory of knowing machines and extend some recent results proving that a machine can know its own code exactly but cannot know its own correctness (despite actually being sound). In particular, we deﬁne a machine that, for (at least) a speciﬁc case, knows its own code and knows to be sound.


Introduction
After the seminal paper by Turing in 1950 [37], the launch of the mechanistic project, intended to establish whether minds can be explained, either extensionally or intensionally, in purely mechanist terms, was contrasted by the socalled Gödelian Arguments. These represent the effort done by several scholars to interpret Gödel's Incompleteness Theorems [12] with the purpose of refuting mechanism. In particular, several speculative ideas, like the anti-mechanist arguments by Lucas [24,25] and Penrose [29], contributed to animate the debate. On the other hand, authors like Benacerraf [5], Chihara [8], and Shapiro [35] tried to follow more sophisticated lines of reasoning for the analysis of the relation between human mind and machines [9]. Most of these approaches preserve intensional elements on properties of human mind that make severely informal the argumentation and, more confusing, make hard even to define precisely what the (anti-)mechanistic thesis claims.
In this setting, a series of results presented by Reinhardt [32], Carlson [7], and Alexander [3], clarify some typically informal aspects of the Gödelian Arguments. This is done in the framework of a theory, called Epistemic Arithmetic (EA), developed independently by both Reinhardt and Shapiro [30,34,31], and encompassing an epistemic notion of intuitive provability. In particular, Reinhardt uses such an axiomatic framework to study both ramifications of the Church-Turing Thesis [21]: every effectively calculable function is computable by a Turing Machine and the consequences of Gödel's Incompleteness Theorems, with the aim of strengthening the formal elements behind the philosophical debate on the knowledge that can be acquired by machines.
EA is the language of Peano Arithmetic enriched by a modal operator K for knowledge, which is the notation used by Shapiro and from Carlson on (see, e.g., [34,7,3]). According to Shapiro, K means "ideally, or potentially, knowable", while Carlson, analogously, says "can eventually come to be known". On the other hand, Reinhardt prefers the more specific interpretation "it is intuitively provable that" [30] and, to strengthen such an idea, uses the epistemic operator B for beweisbar (meaning provable in German). By following the same motivation, we adopt the operator B, which is closer to the intended interpretation of its role, and use the two intuitions -provability and knowledge -interchangeably, by assuming that the notion of knowability [34] is actually limited to intuitive provability. Hence, the idea behind such a modal operator is to express a definition of decidability by human mind (humanly provable) that occurs in many forms both in the mechanistic thesis and in the Gödelian Arguments.
The formal interpretation of B passes through the definition of the properties at the base of an epistemic notion of knowledge. For instance, it is expected that humanly provable statements are closed under logic consequence, meaning that if we can intuitively prove φ → ψ and we have an intuitive proof of φ, then by combining these we derive an intuitive proof of ψ. In other words, we are representing modus ponens as a rule for intuitive provability. Analogously, we also expect the soundness of intuitive provability (what can be proved is true, thus stating the infallibilism of knowledge) and that what is humanly provable includes all tautologies. Using this latter rule in the setting of intuitive provability amounts to establish an introspection principle: if φ can be proved then such a knowledge can be proved. Such a principle may be seen as an instance of the general, largely discussed and controversial, KK (knowing that one knows) law (see, e.g., [17,19]). However, intuitive provability ("it can be proved") is weaker than the general notion of knowledge ("I know that") and relies on idealizations abstracting from space, time, and complexity constraints, thus making the introspection rule less debatable with respect to the classical contrast between internalist and externalist theories of knowledge (see, e.g., [6,15]).
As we will see, all the laws informally expressed above are formalized through very common axioms of epistemic modal systems, see, e.g., [18] for a comprehensive discussion. Therefore, in essence, knowability is treated in axioms form, while any attempt to specify precisely its meaning by following model theoretic approaches is intentionally avoided. In fact, representing knowledge as a predicate easily leads to contradictions forcing all true propositions to be provable [23,27,26,30,32].
Then, the aim of an axioms system based on the operator B is to apply deductive reasoning to prove the (in)consistency of statements specified in an appropriately formulated logic and representing conjectures related to Church's Thesis and the mechanist project, with a specific interest towards the following: the property of being humanly provable is equivalent to decidability by some Turing Machine.
In the rest of the paper, we first illustrate such a theory of knowledge (Section 2). Then, we survey the main results obtained by recasting Gödel Incompleteness Theorems and by analyzing variants of the Church-Turing Thesis in this setting (Section 3). We also show how to extend a tradeoff result obtained by Alexander [3] about the relation between knowledge of soundness and knowledge of own code for (knowing) machines. Some conclusions terminate the paper (Section 4). This paper is a full and revised version of an extended abstract presented at HaPoC 2015 [1].

Epistemic Arithmetic
We start by introducing some notation used to describe the language of Peano Arithmetic (PA) 3 : φ, ψ denote well formed formulas (wff, for short), which, if not specified, are considered to include only one free variable x. 4 A sentence, usually denoted by σ, is a wff without free variables, and any set of sentences closed under logical consequence is called theory. Terms and substitution principle are defined as usual: if t is a term, then x | t denotes the substitution of x by t. An assignment is a function s : V → U from the domain of variables to the reference universe, such that s(x | t) denotes the function assigning t to x and s(y) to every variable y = x.
Epistemic Arithmetic extends Peano Arithmetic with the modal operator B. The language of EA contains every wff of the language of PA and the additional formulas of the form Bφ whenever φ is a wff of EA. In the following, given a set Φ of wff we also use the notation BΦ to denote the set {Bφ | φ ∈ Φ}. It is worth noticing that Bφ is treated as an atomic formula. Then, in EA a structure M with respect to a universe U is defined in the classical way for the first order part related to PA and includes also a boolean interpretation function for B. More precisely, for each assignment s, M |= Bφ[s] means that φ can be intuitively proved when the free variables of φ are interpreted according to s. Obviously, the interpretation of Bφ[s] does not depend on s(x) whenever x does not occur free in φ.
for each assignment s and we say that φ is valid if M |= φ for each structure M; φ is logical consequence of a set of sentences Σ (denoted Σ |= φ) if for each structure M it holds that: We observe that, in such an epistemic extension of PA, compactness and completeness results can be proved. 5 As far as the axiomatization of EA is concerned, we first recall the Peano axioms: , while 3 to 6 define the rules for sum and product, and 7 expresses the induction schema, where φ is a formula with free variables x, y 1 , . . . , y n .
To the standard Peano axioms we add the universal closure of the following basic axioms of knowledge: where B2-B4 formalize the intuitive properties illustrated in Section 1 and are essentially taken from the modal system S4 [22,18]. To complete the axioms system, the first order condition B1 establishes that the statement "φ can be proved to be valid" implies the knowledge of each element that can be assigned to the free variable x of φ and the provability of the formula under each such assignment. This condition represents a sort of accessibility principle ensuring that the elements assigned to the free variable of φ should be accessible to the knower in some way, and from which we notice that the following desirable condition can be derived: which states that if φ can be proved under the interpretation provided by M, then it is possible to prove φ(x) for each assignment of the variable x in the universe U. Now, given a wff φ, we define the B-closure of φ as φ ∪ Bφ, which extends easily to sets of formulas. Then, the standard axiomatization of the theory of knowledge for EA is given by the B-closure of B1-B4 and of the axioms of PA, with the modus ponens being the unique rule of this formal system. The theory of knowledge axiomatized in such a way extends conservatively the classical interpretation of PA.
In particular, let us characterize the following property emphasizing the role of each element of the theory of knowledge. If T is a theory with an axiomatization of the form Σ ∪ BΣ extending the theory of knowledge, then: Hence, we have a correspondence between the valid sentences of T and the sentences that can be intuitively proved in T . On one hand, if σ ∈ Σ ∪ BΣ then, by B4, BBσ ∈ T and, by B3, Bσ ∈ T . Now, assume that σ 2 derives by modus ponens from σ 1 , σ 1 → σ 2 ∈ T . By induction hypothesis, both Bσ 1 and B(σ 1 → σ 2 ) are in T and, by applying B2, we derive Bσ 2 ∈ T . On the other hand, by B3 we have that Bσ ∈ T implies σ ∈ T .
Finally, giving for granted the accessibility principle, it is possible to reason about a simple theory of knowledge [7,3] in which B1 is replaced by the rule of necessitation: if φ, then Bφ. Hence, it can be shown that the theory of knowledge corresponds to the set of sentences that can be proved in the modal system S4.

Knowing Entities and Machines
The statements presented in the next section are considered with respect to a knower reasoning about basic arithmetic. Therefore, we refer to the standard interpretation of the model of arithmetic, extended formally in the following way. Given a set of sentences Σ in EA, the structure N Σ over the set of naturals N is such that, for any wff φ with one free variable x, it holds that: where n is the numeral associated to the natural number n (i.e., the term of the language of PA corresponding to n). Informally, Bφ is satisfied by N Σ under assignment s if and only if φ is logical consequence of Σ whenever replacing each free occurrence of x in φ by the numeral associated to s(x). Then, we say that a theory T in EA is a (knowing) entity if N T is a model of the theory of knowledge. Finally, by using Carlson's notation [7], a recursively enumerable entity is said to be a (knowing) machine.

Properties of knowing machines
While Shapiro was more involved with the description in EA of the grounds of constructive mathematics [34], Reinhardt's main intent was to use the theory of knowledge illustrated above for studying variants of the Church-Turing Thesis. The effective version under analysis is based on a notion of weak decidability, according to which a property φ of natural numbers is weakly decidable if there exists a theorem proving procedure by an idealized human mathematician that can eventually produce each n satisfying the property, i.e., such that φ(n) is true. Such a notion is expressed in EA in terms of the modal operator B, thus leading to a definition of weak B-decidability: ∀n(φ(n) → Bφ(n)), i.e., given a formula φ with one free variable, it holds that the assignments of the variable satisfying φ are known. The objective is then to analyze the relationship between properties that are weakly B-decidable and the Turing Machines (TMs) that formalize the decision algorithm for these properties. Therefore, weak B-decidability expresses in the framework of EA a notion of humanly provable as discussed in Section 1, and its equivalence with respect to Turing computability is represented by the following formula (which we simply call Turing Thesis): where we assume that W e is the recursively enumerable set with Gödel number represented by the PA language term e. Notice that such a statement can be seen as a constructive, effective version of Church's Thesis: a recursively enumerable set exists (and, therefore, a TM enumerates its elements) that contains all and only the assignments of x making φ intuitively provable, that is to say, Turing computability coincides with weak B-decidability. As we will show later, we have that (1) is consistent in EA [30]. However, validity of the Turing Thesis implies that there exists an absolutely undecidable statement. Formally, the following theorem holds.
Theorem 1 (Incompleteness of B). For every theory T in which (1) holds, then there exists φ with one free variable such that: T ∃x(φ(x) ∧ ¬Bφ(x)).
Notice that B3 and (5) imply: ∃e(¬Bφ(e) ∧ ¬B¬φ(e)) because (B¬φ(e) → ¬φ(e)) and φ(e) hold, and therefore it must be ¬B¬φ(e), thus stating the absolute undecidability of φ(e). Such an incompleteness result can be viewed as a version of Gödel's first incompleteness theorem, that is, if T is sound, then it is also incomplete [31]. Analogously, even Gödel's second incompleteness theorem can be recast in this setting to show that the consistency of T is absolutely unprovable. More precisely, a generalized version of Gödel's second incompleteness theorem defined in EA establishes that for any intuitively definable upper bound for intuitive provability we have an absolute impossibility of a consistency proof [31]. Formally, the following second incompleteness of B theorem holds, where we use the notationφ to represent the symbolic expressionn, with n the Gödel number of the formula φ.
Theorem 2 (Incompleteness of B). Assume the existence of a formula ψ with one free variable such that for every sentence σ of T it holds that: then it also holds that: T B¬BConψ where the predicate Con expressing consistency of its argument is defined as ∀x¬(ψ(x) ∧ ¬ψ(x)).
Before proving this theorem, it is worth observing that ψ represents an intuitive upper bound for provability: it is provable that if σ is known, then ψ holds when applied to the symbolic representation of the Gödel number of σ. In fact, notice that the formula above is an instance of the B-closure of B3 stating the knowledge of the bounds of intuitive provability, which are established by ψ. Then, the result states that the consistency of ψ is absolutely unprovable or, in other words, no interesting upper bounds for intuitive provability can be defined. The following proof relies on a preliminary result by Reinhardt [31] stating the applicability of Gödel Fixed Point Lemma 6 in the setting of EA.
This result completes the part concerning the revisiting of the Gödel incompleteness theorems in EA starting from the Turing Thesis. In the following, it is our interest to consider stronger versions of the Turing Thesis involving the knowledge that can be acquired in the theory about such a thesis. The strongest claim, which we call Reinhardt's schema, establishes that the index of the TM deciding Bφ in the Turing Thesis is known. This claim can be refuted in EA, as stated by the following theorem.
Notice that Reinhardt's schema states that a TM exists for which it is known that it enumerates all (and only) the elements (for which it is known) that make φ true. By citing Carlson's intuition, I know to be a TM and I know which one. The inconsistency of this schema can be viewed as an alternative characterization of Gödel's first incompleteness theorem.
Proof. By B1, from Reinhardt's schema we derive: Now, assume φ(x) = ¬(x ∈ W x ) and x = e, hence we have: while by the B-closure of B3: From the conjunction of the two above, by applying tautologies and distributivity, we derive B(φ ∧ ¬Bφ) and then Bφ ∧ B¬Bφ, and applying B3: A weaker version of Reinhardt's schema consists of moving the outermost B operator to prefix the whole formula, thus obtaining the so-called Strong Mechanistic Thesis, which we call Carlson's schema from the author who proved its consistency in EA.
By citing Carlson, I know that the set of x for which I know φ(x) is recursively enumerable or, by rephrasing an hypothesis studied by Benacerraf independently, I know to be a TM but I do not know which one.
To prove Theorem 4, in [7] it is shown that the theory of knowledge plus Carlson's schema is a (knowing) machine. In order to establish the induction proving the result, Carlson generalizes the modal operator B by introducing a collection of operators B t , where t belongs to a linearly ordered set and can be interpreted as the amount of steps (either logical or temporal) needed to prove the formula guarded by B t . In such a stratified version of the theory of knowledge, each formula is such that any occurrence of B t in the scope of another occurrence B t satisfies the condition t < t . Then, the key parts of the proof rely on showing that the standard axiomatization of the stratified theory of knowledge for EA plus Carlson's schema is recursively enumerable and that such a result is inherited by the non-stratified theory.
Theorem 4 implies, as a corollary deriving from the application of B3, the validity of the Turing Thesis. Another interesting corollary of the theorem is related to the first incompleteness of B theorem. Corollary 1. Given T and φ as in Theorem 1, then: T B∃e(φ(e) ∧ ¬Bφ(e)).
Proof. By Theorem 1, (5) holds and if (1) holds whenever prefixed by B, then so does (5), because under such a hypothesis it is possible to prefix with B every step of the proof of Theorem 1. Now, it is sufficient to notice that (1) prefixed by B is exactly Carlson's schema.
Informally, this result states that knowledge of the Turing Thesis implies knowledge that there exists an absolutely undecidable sentence in EA. In particular, the corollary requires B prefixed to B3, which is the only axiom for B used in the proof of (5) in Theorem 1. In general, all the proofs related to variants of the Church-Turing Thesis discussed so far rely on the validity of B(Bφ → φ), stating that in the formal system the soundness (factivity, as called by Alexander) of knowledge can be proved. Under such a condition, we have Theorem 4 stating that for each wff φ, an unspecified TM enumerates recursively the set of values for which one knows that φ is satisfiable (which can be read as "I know to have some code related to φ"). On the other hand, we have Theorem 3 stating that the knowledge of the identity of such a TM cannot be acquired (which can be read as "I cannot know my own code related to φ"). In between the limiting results stated by Reinhardt and Carlson, Alexander has recently proved a dichotomy revealing the relation between knowledge of factivity and knowledge of own code. Either a machine can know to be sound (that is, B(Bφ → φ) is valid) as well as that it has some code (without knowing which, as proved by Carlson), or it can know its own code exactly (thus proving the consistency of Reinhardt's schema) but in such a case cannot know its own soundness (that is, B(Bφ → φ) is not valid anymore). As a consequence, we emphasize that renouncing knowledge of soundness implies that the machine loses also knowledge of incompleteness.
Providing that the axioms of EA mod factivity consist of the axioms of EA except for the universal closure of B3 prefixed by B (i.e., B(Bφ → φ) is not valid), it is possible to prove that: Theorem 5 (Alexander [3]). Reinhardt's schema is consistent in EA mod factivity. and then to construct the previous dichotomy.
In order to sketch the main intuition behind the proof (see [2] for details), it is worth considering the family of axioms Σ(n), for n ∈ N, which essentially consists of the axioms of EA mod factivity minus B3 and plus the additional schemes: ∀x(Bφ ↔ b(x,φ) ∈ Wn) for any wff φ with one free variable x where b is a canonical computable bijection -hence definable in the language of PA -mapping pairs (represented by the symbolic Gödel number of φ and its input x) to numerals. Since by completeness and compactness of EA, Σ(n) is recursively enumerable, then, by the Church-Turing Thesis, chosen φ as above, for every n ∈ N there exists a total computable function f : N → N such that: meaning that the set of elements that, assigned to x in φ, make φ a logical consequence of Σ(n), is recursively enumerable by a set of index f (n). Now, by Kleene's Recursion Theorem, it holds that there exists e ∈ N such that W e = W f (e) .
Summarizing, we have described a recursively enumerable set of sentences (including EA mod factivity and Reinhardt's schema) that turns out to be an entity, which represents the result expressed by Theorem 5.
In the framework of such a machine, we show a result related to a specific case. Consider φ(x) = (x ∈ W x ) and x = e. Thus from: we derive: ∃eB(Bφ(e) ↔ e ∈ W e ) and: which expresses a limited form of knowledge of soundness allowed in EA mod factivity. More precisely, we have a specific function φ for which the related machine knows its own code and knows to be sound with respect to a specific input x. We notice that, by definition, φ turns out to represent an instance of an interpreter function 7 and the specific input x is the index of the machine itself. Therefore, by following the same intuitions derived from Reinhardt and Carlson theorems, "If I am a universal TM knowing my own code, then I know the soundness of the computation provided when I am fed with my own code". By virtue of the previous results and according to Alexander's dichotomy, knowledge about other TMs is much more limited. As a consequence, in general a universal TM knowing its own code cannot know the soundness of what it proves when interpreting other TMs. On the other hand, if a (universal) TM can prove the soundness of its knowledge, then it cannot know its own code. In our opinion, this is an interesting enhancement of the tradeoff result provided by Alexander representing an additional formal element for the analysis of the Gödelian Arguments.

Conclusion
Historically, the idea of a provability logic is first discussed by Gödel [13,14] with the aim of defining a formal semantics for intuitionistic truth [36]. Gödel's calculus is based on propositional logic and on the modal operator 2, and is basically equivalent to S4 [22]. The resulting notion of provability shall not be confused with formal deducibility for theories including PA, as such a correspondence leads to a contradiction [14]. Following Gödel's attempt, several approaches have been proposed either to define axiom systems for formal deducibility, or to find an exact provability semantics for S4-like modal systems. For a comprehensive survey, we refer, e.g., to [4].
In this setting, the theory of knowing machines offers a proof-theoretic framework to reason about the notions of intuitive provability and consistency of TMs. In practice, the extended results show precisely the relation between selfawareness of soundness and of own code in the setting of knowing machines, and show some compatibility with philosophical arguments like the following suggestions due to Gödel himself [38]: On the other hand, on the basis of what has been proved so far, it remains possible that there may exist (and even be empirically discoverable) a theorem proving machine which in fact is equivalent to mathematical intuition, but cannot be proved to be so, nor even be proved to yield only correct theorems of finitary number theory.
The results provided in EA are obtained by following formal lines of reasoning even if starting from the definition of an epistemic notion of intuitive provability. In fact, on one hand, it is worth noticing that the modal operator B necessitates an atomic treatment of formulas of the form Bφ and allowing only for an axiomatic representation of its properties. Hence, the lack of a precise modeltheoretic semantics represents a weakness. However, on the other hand, it is also worth observing that all the proofs of the results provided by Reinhardt, Carlson, and ourselves derive from the formal application of deductive reasoning and proof theoretic techniques. The unique exception is given by Theorem 5, which relies on the application of the Church-Turing Thesis (see, as an example, [33] for the informal aspects of practical uses of the Church-Turing Thesis).
Even in the setting of EA as defined by Shapiro [34], the consistency of epistemic variants of Myhill's version of the Church-Turing Thesis [28] can be demonstrated [11,16,20]. Such variants are captured by the following informal statement: if it is intuitively provable that for each x there exists y such that it holds that φ(x, y) can be proved, then φ determines a total recursive function.