This is the editors' words from Haoran and Yuyang. They want to explain their ideas of creating this website. If you find what you want here, or just simply like it, please consider giving a star to its repo on GitHub!

Why Simulately?

At the intersection of robotics and computer vision, physics simulators serve as crucial tools for validating algorithms and testing models before they're deployed in real-world robots. They cater to a wide range of functions, from rendering photorealistic perceptions with ground truth segmentation masks, to spawning parallel environments with GPU for efficient data sampling in Reinforcement Learning. Consequently, an array of physics simulators has emerged, each with its unique set of offerings. However, several challenges persist:

1. Some of the required functions are found missing on the half way of development with an initially chosen simualtor.
2. Simulation results are inconsistent across different simulators, adding another layer of complexity.
3. Some simulators suffer from poor documentation, leaving users in the dark about how to implement the desired functions using the provided APIs.
4. Troubleshooting can be a nightmare, with solutions either hard to come by, incomplete or entirely absent from online resources.
5. The list goes on and on and on and on... Don't stop believing

Addressing these hurdles for the benefit of the wider community, particularly researchers who could better spend their time on more meaningful pursuits, we've created Simulately. Our mission extends beyond creating a repository for our experiences in research and development. We aim to foster a collaborative platform where learnings and insights are shared, and where others are encouraged to do the same. Welcome to Simulately, your comprehensive guide to navigating the world of robotics simulators.

In this blog, we discuss what is physics simulator and why it is important for robotics. Some text gets help from GPT-4. If you find what you want here, or just simply like it, please consider giving a star to its repo on GitHub!

What is a Simulator?

A physics simulator is a computer program or software that models the behavior of physical systems according to the laws and principles of physics. These simulators are used in a wide range of applications, from video games and movies to engineering and scientific research. The goal of a physics simulator is to predict or replicate how physical objects interact and evolve over time in a given environment.

There are several key features and components commonly found in physics simulators:

1. Collision Detection and Response: Determines when two or more objects collide and dictates how they should respond, such as bouncing off each other or deforming upon impact.
2. Rigid Body Dynamics: Simulates the motion of solid objects that don't change shape. This includes things like velocity, acceleration, and forces like friction and gravity.
3. Soft Body Dynamics: Simulates the motion and deformation of objects that are not rigid, like cloth or jelly.
4. Particle Systems: Simulates systems of many small particles, which could be anything from water spray to smoke or fire.
5. Fluid Dynamics: Models the behavior of liquids and gases, capturing phenomena like flow, turbulence, and wave propagation.
6. Constraint Solvers: Manage constraints or restrictions on how objects can move, like a door that can swing only in one direction.

Different physics simulators may emphasize or specialize in one or more of these areas depending on their intended application. For instance:

• Video games might use simplified or approximated physics to ensure fast real-time performance.
• Animation software for films might have detailed soft body dynamics to model realistic clothing or hair movement.
• Engineering software might focus on accurate rigid body dynamics to predict the behavior of machinery or structures.

Using physics simulators, researchers and developers can test scenarios in a virtual environment before implementing them in the real world, which can save time, money, and reduce risks.

Why Physics Simulator is crucial for Robotics?

In the field of robotics, physics simulators play an indispensable role. They allow engineers and researchers to model and test robotic systems in a controlled virtual environment before physical prototypes are built. This is particularly crucial in robotics due to the complexity and cost of robotic systems. Simulators provide a safe and cost-effective way to explore the behavior of robots in various scenarios, including challenging or hazardous environments. By using these tools, developers can optimize the design and functionality of robots, ensuring that they perform as intended in the real world. This includes testing the robot's ability to navigate terrain, manipulate objects, and interact safely and effectively with humans and other machines. Furthermore, simulators are essential for training artificial intelligence systems in robotics, offering a diverse range of scenarios for machine learning algorithms to learn from, without the risks and costs associated with real-world testing.

What we want to do?

Our project is designed to cater to both beginners and seasoned developers in the field of robotics and simulation. For beginners, we aim to create a single, comprehensive website that serves as a one-stop resource. This platform will feature easy-to-follow tutorials for those just starting their journey in physics simulations and robotics. Additionally, we plan to offer a series of blogs that break down complex concepts into digestible, easy-to-learn formats, fostering a friendly learning environment for newcomers.

For the more experienced developers, our project takes a deeper dive. We intend to provide a thorough summary of related works and benchmarks in the field, giving professionals a quick yet comprehensive overview of the current state of the art. Our platform will also feature a collection of useful toolkits, designed to streamline the development process and enhance efficiency. Furthermore, a key component of our project is a deep comparison of all available simulators, offering detailed insights and evaluations to assist developers in choosing the right tools for their specific needs.

In essence, our goal is to bridge the gap between beginners and experts in the world of robotics and physics simulations, creating a harmonious community where knowledge and resources are shared efficiently and effectively.

In this blog, we delve into the mechanics of differentiable simulators and explore why they are sometimes a more advantageous choice compared to reinforcement learning (RL) methods.

What is a Differentiable Simulator?​

Imagine a robot in an environment where, in each state $s \in \mathcal{S}$, the agent can execute an action $a \in \mathcal{A}$ leading to a subsequent state $s' \in \mathcal{S}$. We can describe this transition with the function $f: \mathcal{S} \times \mathcal{A} \to \mathcal{S}$. In conventional non-differentiable simulators, this function $f$ is often treated as a black box, with observed rewards $r \in \mathcal{R}$ serving as the primary signal for RL-based learning.

Contrastingly, in differentiable simulators, the function $f$ is perceived as an end-to-end differentiable operator. This implies that if we define some loss related to the output state $l(s')$, it becomes feasible to compute the gradient in relation to the input state and actions, such as $\frac{\partial l(s')}{\partial s}$ and $\frac{\partial l(s')}{\partial a}$. This capability enables the optimization of an entire sequence of actions using the chain rule.

Why are Differentiable Simulators Effective for Policy Learning?​

As Yann LeCun insightfully noted at NeuIPS 2016, "If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning" (source). The key takeaway here is the direct supervisory gradient flow in relation to the actions we aim to optimize, in contrast to the indirect reward-based approach of the REINFORCE algorithm in RL.

For instance, consider a task where the goal is to maneuver an object to a target position using a pusher. In an RL scenario, this would require extensive exploration, potentially involving thousands of trials before significant progress is made. Conversely, in a differentiable simulator, each iteration of gradient descent inherently contributes to progress toward the goal.

Miscellaneous Considerations​

While differentiable simulators present certain advantages over RL, they are not without their limitations and challenges, such as:

• Invalid Gradients in Certain Scenarios: Consider a scenario involving a rigid rolling pin used to flatten dough. If the initial sequence of actions fails to make contact with the dough, the resulting gradient will be zero throughout. To address this, some studies, like PlasticineLab, suggest incorporating a contact loss based on proximity to the target object. Others propose 'softening' rigid tools by increasing their influence radius, allowing objects to be affected without direct contact.

• Limited Efficiency in Long-Horizon Tasks: As discussed in this paper, the dependency of differentiable physics on local gradients poses significant challenges. The loss landscape in these scenarios is often complex and riddled with potentially misleading local optima, which can diminish the reliability of this method for certain tasks.