Shiro Takagi

About

I am an independent researcher on intelligence. My long-term research goal is to create an artificial researcher. I am interested in symbolic fluency, memory, and autonomy.

I became an independent researcher to contribute to developing research ecosystems outside the existing system. I'd love to discuss with more metascience entrepreneurs* sharing the vision with me.

I completed my master of research at the Department of Complexity Science and Engineering at the University of Tokyo. Prior to that, I received my B.A. in Economics at Keio University, where I studied economics, international relations, and neuroaesthetics.

* Michael Nielsen and Kanjun Qiu, 2022, "A Vision of Metascience"

Research

I'm currently interested in how connectionist models can segment the world into symbol-like discrete primitives, form the symbolic system, and use the symbols to behave and think like humans. One of my ongoing projects is about systematic/compositional generalization. In the past I have been involved in research on the following topics:

Pre-Training in NLP
Offile RL
NNGP (Neural Network Gaussian Process)
Topological data analysis
Continual learning
Meta-learning
Out-of-distribution genralization
Trainability of neural networks
Disentangled representation
Statistical mechanical analysis of learning dynamics of neural networks
Sense of agency

Publications

Empirical Study on Optimizer Selection for Out-of-Distribution Generalization

Hiroki Naganuma, Kartik Ahuja, Ioannis Mitliagkas, Shiro Takagi, Tetsuya Motokawa, Rio Yokota, Kohta Ishikawa, Ikuro Sato

arXiv:2211.08583
On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning

Shiro Takagi

Advances in Neural Information Processing Systems 35 (2022)

Code / LaTeX / Poster
Convergence of Neural Networks to Gaussian Mixture Distribution

Yasuhiko Asao, Ryotaro Sakamoto, Shiro Takagi

arXiv:2204.12100

Code
Image Recognition via Vietoris-Rips Complex

Yasuhiko Asao, Jumpei Nagase, Ryotaro Sakamoto, Shiro Takagi

arXiv:2109.02231

Code
Statistical Mechanical Analysis of Catastrophic Forgetting in Continual Learning with Teacher and Student Networks

Haruka Asanuma, Shiro Takagi, Yoshihiro Nagano, Yuki Yoshida, Yasuhiko Igarashi, Masato Okada

arXiv:2105.07385
Analysis of Trainability of Gradient-based Multi-environment Learning from Gradient Norm Regularization Perspective

Shiro Takagi, Yoshihiro Nagano, Yuki Yoshida and Masato Okada

The International Joint Conference on Neural Networks 2021, Shenzhen, China, July 18-22, 2021
The Effect of Batch Normalization in the Symmetric Phase

Shiro Takagi, Yuki Yoshida and Masato Okada

29th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 15-18, 2020
Impact of Layer Normalization on Single-Layer Perceptron ― Statistical Mechanical Analysis

Shiro Takagi, Yuki Yoshida and Masato Okada

Journal of the Physical Society of Japan, 88(7), 2019
What Makes Action and Outcome Temporally Close to Each Other: A Systematic Review and Meta-Analysis of Temporal Binding

Takumi Tanaka, Takuya Matsumoto, Shintaro Hayashi, Shiro Takagi and Hideaki Kawabata

Timing & Time Perception, 8(3), 2019
Re-Examination International Public Goods Theory of U.S. Japan Alliance (in Japanese)

Shiro Takagi

Seijigaku Kenkyu - Studies in Politics, 59, 2017 (Non-refereed)

Talks/Workshops

Managing the Whole Research Process on GitHub

Shiro Takagi

NeurIPS Workshop on Broadening Research Collaborations in ML, 2022

LaTeX / Poster
Separation of Research Data from Its Presentation

Shiro Takagi

NeurIPS Workshop on Broadening Research Collaborations in ML, 2022

LaTeX / Poster
Thoughts on the Applicability of Machine Learning to Scientific Discovery and Possible Future Research Directions (Perspective)

Shiro Takagi

3rd AI4Science Workshop @ NeurIPS, 2022

LaTeX / Poster
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization

Hiroki Naganuma, Kartik Ahuja, Ioannis Mitliagkas, Shiro Takagi, Tetsuya Motokawa, Rio Yokota, Kohta Ishikawa, Ikuro Sato

NeurIPS 2022 Workshop on Distribution Shifts, 2022
On The Conditions of MAML Convergence

Shiro Takagi, Yoshihiro Nagano, Yuki Yoshida, Masato Okada

NeurIPS Workshop on Meta-Learning, 2019
Localized Generations with Self-Supervised Meta-Learning

Yoshihiro Nagano, Shiro Takagi, Yuki Yoshida, Masato Okada

NeurIPS Workshop on Learning with Rich Experience: Integration of Learning Paradigms, 2019
The Effect of Learning Rates on Model-agnostic Meta-learning

Shiro Takagi, Yoshihiro Nagano, Yuki Yoshida, Masato Okada

ACML Workshop on Statistics & Machine Learning Researchers in Japan, 2019
Learning Locally Structured Dataset with Gradient-Based Meta-Learning

Yoshihiro Nagano, Shiro Takagi, Yuki Yoshida, Masato Okada

ACML Workshop on Statistics & Machine Learning Researchers in Japan, 2019
Statistical Mechanical Formulation of Learning Dynamics of Two-Layered Neural Networks with Batch Normalization

Shiro Takagi, Yuki Yoshida and Masato Okada

The 33rd Annual Conference of the Japanese Society for Artificial Intelligence, 2019
The Impact of Layer Normalization on Single-Layer Perceptron — Statistical Mechanical Analysis

Shiro Takagi, Yuki Yoshida and Masato Okada

The Physical Society of Japan Annual Meeting, 2019
Statistical Mechanical Analysis of Single-layer Perceptron with Layer Normalization

Shiro Takagi, Yuki Yoshida and Masato Okada

The 21st Information-Based Induction Sciences Workshop, 2018

Misc

Podcast

metascientia (in Japanese)

Survey

Articles

Talks

Towards Creating and Aritifial Agent to Acquire a Language (in Japanese) @RMCII2022