Posts

How is model size measured for Large Language Models?

AI Scaling  ChatGPT4 gives a far better answer than Gemini/Bard, although Chat's outdated training cutoff in 'early 2023' is a handicap. Hoffman et al. found " that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of model size the number of training tokens should also be doubled. "  One of the authors, Arthur Mensch, has founded a company, Mistral , based on making the most efficient AI components and advancing Europe in the AI arms race (an expression I don't care for because it conflicts with safety). ChatGPT4 Model size for Large Language Models (LLMs), including GPT models, is typically measured by the number of parameters they have. These parameters are the internal settings learned from data during the training process, and they dictate how the model responds to input queries. A parameter in this context is a component of the model's architecture that can adjust and learn fr...

A general-purpose program to verify software functionality is undecidable

What sorts of problems are undecidable by computer? Are they esoteric, dwelling only in the minds of theoreticians? No! Even some ordinary problems that people want to solve turn out to be computationally unsolvable. In one type of unsolvable problem, you are given a computer program and a precise specification of what that program is supposed to do (e.g., sort a list of numbers). You need to verify that the program performs as specified (i.e., that it is correct). Because both the program and the specification are mathematically precise objects, you hope to automate the process of verification by feeding these objects into a suitably programmed computer. However, you will be disappointed. The general problem of software verification is not solvable by computer. - Sipser 2006 p. 201 I can see two strategies to prove this important theorem whose importance is increased when we think about its implications in proving artificial general intelligence (AGI) safety or value alignment. First,...

Methods of Proving Undecidability

 ChatGPT4 There are undecidability proofs that do not rely directly on the reducibility of the Halting Problem. These proofs often demonstrate the undecidability of a problem through other means, such as diagonalization, which was famously used by Cantor to prove that the set of real numbers is uncountable, and later adapted by Turing for his original undecidability proof of the Halting Problem itself. Here are a few notable examples and approaches: 1. **Diagonalization**: This method, used by Alan Turing in his original paper proving the undecidability of the Halting Problem, can also be applied directly to prove the undecidability of other problems. The essence of diagonalization is to assume that a complete list (or enumeration) of all possible instances of something (e.g., programs, functions) exists, and then construct a new instance that contradicts the completeness of that list. This technique can be adapted to show that certain problems cannot be decided by any algorithm....

Proof of Undecidability of Post Correspondence Principle

ChatGPT4: The Post Correspondence Problem (PCP) is a classic decision problem in the field of computability theory and theoretical computer science, formulated by Emil Post in 1946. The essence of why PCP is undecidable lies in its connection to the concept of algorithmic undecidability, which means there is no general algorithm that can solve all instances of the problem. The problem can be described as follows: Given two lists of strings, A = [ a 1 , a 2 , ... , a n ] and B = [ b 1 , b 2 , ... , b n ], over some alphabet, the question is whether there exists a finite sequence of indices [i 1 , i 2 , ... , i k ] with 1 ≤ i j ≤ n for each j, such that the concatenation of the strings from list A using this sequence of indices is equal to the concatenation of the strings from list B using the same sequence of indices. That is, we want to know if [ a i1 , a i2 , ... , a ik ] and B = [ b i1 , b i2 , ... , b ik ], The undecidability of the Post Correspondence Problem comes fro...

How does the earth's magnetic field protect life from incoming radiation?

Image
 The Earth's magnetic field plays a crucial role in protecting life on our planet from incoming solar and cosmic radiation. This magnetic shield, also known as the magnetosphere, extends far into space and serves as a protective barrier against the harmful effects of solar wind and cosmic rays. Here's how it works: 1. **Deflection of Solar Wind**: The solar wind, a stream of charged particles (mainly electrons and protons) emitted by the Sun, carries with it a significant amount of energy and harmful radiation. When these charged particles encounter Earth's magnetic field, their paths are altered due to the Lorentz force, which acts perpendicular to their velocity and the magnetic field lines. This deflection largely prevents the solar wind from reaching the Earth's surface. Van Allen radiation belts. The Van Allen radiation belts contained within Earth's magnetosphere. Pressure from the solar wind is responsible for the asymmetrical shape of the magnetosphere and t...

How do I adjust your stochastic element? I want to reduce 'hallucinations' in some of your answers.

 ChatGPT-4 As of my last update in January 2022, users cannot directly adjust the "stochastic element" of my responses. I'm designed to generate text based on a set of weights and parameters that are fixed during the training process. Any random behavior or "hallucinations" are characteristics of the model and are not configurable by end-users. However, OpenAI does have methods for fine-tuning and adjusting various aspects of the model during the training and deployment process, but these capabilities are typically only available to researchers and developers working directly with the model's code and data. If you're finding that the answers I provide contain inaccuracies or hallucinations, your best course of action may be to rephrase your question or ask for clarification. This can often lead to more accurate and useful answers. Q:  Is there a way to tune the prompt, as you say, to indirectly control the temperature or top-p? Yes, you can attempt to in...

So you can do translation?

 Yes, I can perform translation tasks between multiple languages, although my translations may not be as accurate as those from specialized translation services. My capabilities are suitable for general understanding and communication but might not be sufficient for highly technical or legal documents that require precise translation. Feel free to provide text you'd like translated, and I'll do my best to assist you. Can you teach language? Certainly, I can help you learn a new language by providing explanations, examples, and exercises. I can assist with vocabulary building, grammatical concepts, sentence structure, and even conversational practice to some extent. Whether you're a beginner looking to grasp the basics or an intermediate learner aiming to improve, I can tailor my guidance to suit your needs. However, it's important to note that while I can provide valuable supplementary material and practice, becoming proficient in a new language usually involves a comb...