Big AI Dream | Jie Fu

Sat, 03 Jan 2026 00:00:00 +0000

A Definition of AGI

Wed, 01 Oct 2025 00:00:00 +0000

Re:Form -- Reducing Human Priors in Scalable Formal Software Verification with RL in LLMs: A Preliminary Study on Dafny

Wed, 23 Jul 2025 00:00:00 +0000

Autoformalization and Formally Verifiable AI

Sun, 25 May 2025 00:00:00 +0000

Motivation & Goal

Autoformalization (converting natural language content into verifiable formalization) is considered a promising way of learning general purpose reasoning¹. In sharp contrast, existing natural language-based LLMs lack reliable verification, e.g., there seems to be no free lunch for inference scaling², which is believed to be able to boost LLMs’ reasoning capabilities, as the verifier (reliable training signals) is not perfect for most cases. Formal verifiers are not only important for increasing the resiliency of humanity³, but also vital for steering the AI development into a direction I call maximally math-seeking, which could be, hopefully, more human-friendly and realistic.

Formal verification is usually extremely hard to obtain, and recent progress in automated reasoning could potentially make this approach easier. However, existing LLMs cannot do genuine logical reasoning or self-verification on their own, and they should be viewed as universal approximate knowledge retrievers (e.g., trying to mimic the reasoning steps seen during training)⁴. Given the important role of formal verifiers, I’m exploring ways of scaling up them.

A more detailed blog post is here

Approaches

We are focusing on the following approaches (not necessarily exclusive):

Deep reinforcement learning
Generative model, e.g., diffusion LM, autoregressive LM
Formal verification

We focus on scaling things up!

Seeking Collaboration

If you are a full-time researcher (familiar with formal verification, software engineering, or deep RL) and is interested in collaborating, I’d be very happy to chat.
If you are a student (eager to learn formal verification, software engineering, and deep RL – better already familiar with one of these), I could host you as an intern. See this page

Credits

Since this is a big and ambitious project, I believe that it is important to assign the correct amount credits to each member. I carefully divide it into a series of mini-projects. Each member is encouraged to take a lead in one of them, and each mini-project would hopefully result in an independent paper; More importantly, these mini-projects will be merged as the project to solve a big problem.

https://research.google/pubs/a-promising-path-towards-autoformalization-and-general-artificial-intelligence/ ↩︎
https://arxiv.org/abs/2411.17501 ↩︎
The plan described by ARIA could potentially greatly increase the safety level but seems a bit over-constraining, which might introduce cancerous states as discussed in this paper. ↩︎
https://x.com/rao2z/status/1740692722099630237 ↩︎

Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking

Thu, 27 Feb 2025 00:00:00 +0000

Unlocking Emergent Modularity in Large Language Models

Sat, 29 Jun 2024 02:00:00 +0000

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Thu, 30 May 2024 00:00:00 +0000

Tracking single cell evolution via clock-like chromatin accessibility

Mon, 20 May 2024 02:00:00 +0000

Think Before You Act: Decision Transformers with Working Memory

Sun, 19 May 2024 02:00:00 +0000

Massive Editing for Large Language Models via Meta Learning

Tue, 16 Jan 2024 02:00:00 +0000

Tue, 24 Oct 2023 00:00:00 +0000

Unifying Likelihood-free Inference with Black-box Sequence Design and Beyond

Thu, 20 Jan 2022 00:00:00 +0000

Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters

Fri, 02 Apr 2021 00:00:00 +0000

Our paper received the Outstanding Paper Award (8 out of 860 accepted papers).

Mentoring Junior Students

Sun, 08 Nov 2020 00:00:00 +0000

I 😹 am always passionate about talking / mentoring / working with junior self-motivated students. I prefer writing detailed documents so that team members can discuss the technical details more easily.

Drop me an email if you want to have a chat.

I feel honored to be able to work, and grow together, with many young students with unlimited talent. Here is a list of students that I am co-mentoring or co-mentored with other senior researchers:

Current

Zixuan Liu, Drug Discovery, PhD Student at University of Washington
Dan Liu, Meta Learning, PhD Student at McGill University
Jikun Kang, Meta RL , PhD Student at McGill University
Shaoxiong Ji, Protein LM, PhD Student at Aalto University
Osana Ratnaharan, Health, Undergraduate Student at University of Toronto
Qifeng Wu, Drug Discovery, Master Student at Fudan University
Maolong Yang, Meta Learning, Undergrad Student at Tsinghua University
Zedian Xiao, Drug Discovery, Undergrad Student at McGill University
Haodong Ling, Drug Discovery, Undergrad Student at Fudan University

2020

Bingchan Zhao, RL for power network, undergraduate at Peking University master at Peking University
Zhijian Duan, RL, undergraduate at Peking University PhD at Peking University

2019

Mustafa Alghali, AI for social good, master at AMMI data scientist at Unity
Ronak Pradeep, Question answering, undergraduate at University of Waterloo master at University of Waterloo
Ning Dai, Translation, undergraduate at Fudan University research intern at ByteDance

2018

Dayiheng Liu, Text generation, PhD at Sichuan University research intern at Microsoft Research Asia
Shangbang Long, Adversarial attack, undergraduate at Peking University master at CMU
Vardaan Pahuja, Visual question answering, master at Mila PhD at Ohio State University

2017

Danlu Chen, Hyperparameter optimization, undergraduate at Fudan University PhD at UCSD
Ritchie Ng, Hyperparameter optimization, undergraduate at NUS hedge fund

2016

Zichuan Lin, RL, undergraduate at Tsinghua University PhD at Tsinghua University

2015

Hongyin Luo, Hyperparameter optimization, undergraduate at Tsinghua University PhD at MIT

微信

Sun, 08 Nov 2020 00:00:00 +0000

Please indicate that you find my account through my homepage.

Mon, 01 Jan 0001 00:00:00 +0000