In this blog, we discuss what is physics simulator and why it is important for robotics. Some text gets help from GPT-4. If you find what you want here, or just simply like it, please consider giving a star to its repo on GitHub!

What is a Simulator?

A physics simulator is a computer program or software that models the behavior of physical systems according to the laws and principles of physics. These simulators are used in a wide range of applications, from video games and movies to engineering and scientific research. The goal of a physics simulator is to predict or replicate how physical objects interact and evolve over time in a given environment.

There are several key features and components commonly found in physics simulators:

1. Collision Detection and Response: Determines when two or more objects collide and dictates how they should respond, such as bouncing off each other or deforming upon impact.
2. Rigid Body Dynamics: Simulates the motion of solid objects that don't change shape. This includes things like velocity, acceleration, and forces like friction and gravity.
3. Soft Body Dynamics: Simulates the motion and deformation of objects that are not rigid, like cloth or jelly.
4. Particle Systems: Simulates systems of many small particles, which could be anything from water spray to smoke or fire.
5. Fluid Dynamics: Models the behavior of liquids and gases, capturing phenomena like flow, turbulence, and wave propagation.
6. Constraint Solvers: Manage constraints or restrictions on how objects can move, like a door that can swing only in one direction.

Different physics simulators may emphasize or specialize in one or more of these areas depending on their intended application. For instance:

• Video games might use simplified or approximated physics to ensure fast real-time performance.
• Animation software for films might have detailed soft body dynamics to model realistic clothing or hair movement.
• Engineering software might focus on accurate rigid body dynamics to predict the behavior of machinery or structures.

Using physics simulators, researchers and developers can test scenarios in a virtual environment before implementing them in the real world, which can save time, money, and reduce risks.

Why Physics Simulator is crucial for Robotics?

In the field of robotics, physics simulators play an indispensable role. They allow engineers and researchers to model and test robotic systems in a controlled virtual environment before physical prototypes are built. This is particularly crucial in robotics due to the complexity and cost of robotic systems. Simulators provide a safe and cost-effective way to explore the behavior of robots in various scenarios, including challenging or hazardous environments. By using these tools, developers can optimize the design and functionality of robots, ensuring that they perform as intended in the real world. This includes testing the robot's ability to navigate terrain, manipulate objects, and interact safely and effectively with humans and other machines. Furthermore, simulators are essential for training artificial intelligence systems in robotics, offering a diverse range of scenarios for machine learning algorithms to learn from, without the risks and costs associated with real-world testing.

What we want to do?

Our project is designed to cater to both beginners and seasoned developers in the field of robotics and simulation. For beginners, we aim to create a single, comprehensive website that serves as a one-stop resource. This platform will feature easy-to-follow tutorials for those just starting their journey in physics simulations and robotics. Additionally, we plan to offer a series of blogs that break down complex concepts into digestible, easy-to-learn formats, fostering a friendly learning environment for newcomers.

For the more experienced developers, our project takes a deeper dive. We intend to provide a thorough summary of related works and benchmarks in the field, giving professionals a quick yet comprehensive overview of the current state of the art. Our platform will also feature a collection of useful toolkits, designed to streamline the development process and enhance efficiency. Furthermore, a key component of our project is a deep comparison of all available simulators, offering detailed insights and evaluations to assist developers in choosing the right tools for their specific needs.

In essence, our goal is to bridge the gap between beginners and experts in the world of robotics and physics simulations, creating a harmonious community where knowledge and resources are shared efficiently and effectively.

Physical simulations are a crucial tool in many fields, from game development and computer graphics to robotics and prototype modeling. One fundamental aspect of these simulations is the concept of rotation. Be it planets whirling around a star in a space simulation, joints operating in a humanoid robot, or an animated character performing a thrilling parkour backflip, rotations are indeed everywhere. This blog post seeks to unravel the complexities of 3D rotations and acquaint you with the diverse rotation representations used in physical simulations.

Challenges of 3D Rotations​

3D rotations are crucial for modeling the orientation of objects in space. They enable us to visualize and manipulate 3D models mathematically. However, handling rotations in 3D space can be quite tricky. Many bugs in simulations can be traced back to mismanaged rotations. The complexities arise from the nature of the 3D rotation itself – it isn't commutative (the sequence of rotations is crucial) and interpolation isn't straightforward ( calculating a rotation halfway between two given rotations is complex). Additionally, 3D rotations form a group structure known as the Special Orthogonal Group, SO(3), which isn't a typical Euclidean space where we can perform standard linear operations.

Rotation Representations​

1. Rotation Matrices​

Rotation matrices are 3x3 matrices that signify a rotation around the origin in 3D space. They provide an intuitive approach to understanding rotation, with each column (or row, depending on convention) of the matrix representing the new directions of the original axes after the rotation.

However, rotation matrices come with their set of limitations. The degree of freedom for rotation in an n-dimensional space is $\frac{n(n-1)}{2}$. Thus, the 3D rotation resides in a 3-dimensional space (while 2D rotation resides in a 1-dimensional space). This means that 3D rotation matrices consume more memory (9 floating point numbers) than necessary, and maintaining the orthogonality and normalization of the rotation matrix during numerical operations can be computationally burdensome. In practical applications, the majority of libraries, including the simulators we've discussed, employ quaternions as their core representation for rotations.

2. Quaternions​

Quaternions are a type of mathematical object that extend complex numbers. They consist of one real component and three imaginary components, often denoted as $w+xi+yj+zk$. Quaternions have emerged as an extremely effective method of representing rotations in 3D space for computation.

Different from rotation matrix, they merely require four floating point numbers, can be easily interpolated using techniques like Spherical Linear Interpolation (SLERP), and they bypass the gimbal lock problem. However, they are not as intuitive as the other methods, and comprehending how they work necessitates some mathematical background. Also, quaternions have a double covering problem. This means that each 3D rotation can be represented by two different quaternions: one and its negation. In other words, a quaternion $q$ and its negative $-q$ will represent the same 3D rotation.

3. Euler Angles​

Euler angles represent a rotation as three angular rotations around the axes of a coordinate system. The axes can be in any order (XYZ, ZYX, etc.), and this order makes a difference, leading to what is known as the "gimbal lock" problem.

Gimbal lock occurs when the axes of rotation align, causing a loss of one degree of freedom. This can lead to unexpected behavior in simulations. And same 3D rotation can be mapped into multiple Euler angles. Euler angles also have issues with interpolation, as interpolating between two sets of Euler angles will not produce a smooth rotation.

4. Axis-Angle Representation​

The Axis-Angle representation is another way to understand 3D rotations. In this representation, a 3D rotation is characterized by a single rotation about a specific axis. The amount of rotation is given by the angle, and the direction of rotation is specified by the unit vector along this axis.

This representation is simple and intuitive, but it's not easy to concatenate multiple rotations. Also, like Euler angles, it has a gimbal lock problem when the rotation angle reaches 180 degrees. However, it's very useful in some scenarios such as generating a random rotation, or rotating an object around a specific axis.

Conversion Between Representations​

Now, let's discuss the conversion between a rotation matrix and other common rotation representations: Euler angles, quaternions, and the axis-angle representation.

1. Rotation Matrix to Euler Angles​

The process of extracting Euler angles from a rotation matrix depends on the Euler angles convention. For the XYZ convention (roll, pitch, yaw), the extraction is:

roll = atan2(R[2, 1], R[2, 2])
pitch = atan2(-R[2, 0], sqrt(R[0, 0]^2 + R[1, 0]^2))
yaw = atan2(R[1, 0], R[0, 0])

where $R[i, j]$ denotes the element at the $i$-th row and the $j$-th column of the rotation matrix $R$.

2. Rotation Matrix to Quaternions​

The conversion from a rotation matrix $R$ to a quaternion $q = (w, x, y, z)$ can be computed as:

w = sqrt(1 + R[0, 0] + R[1, 1] + R[2, 2]) / 2
x = (R[2, 1] - R[1, 2]) / (4 * w)
y = (R[0, 2] - R[2, 0]) / (4 * w)
z = (R[1, 0] - R[0, 1]) / (4 * w)

3. Rotation Matrix to Axis-Angle​

For converting a rotation matrix to axis-angle representation, the axis $a = (a_x, a_y, a_z)$ and the angle $\theta$ can be calculated as:

θ = acos((trace(R) - 1) / 2)
a_x = (R[2, 1] - R[1, 2]) / (2 * sin(θ))
a_y = (R[0, 2] - R[2, 0]) / (2 * sin(θ))
a_z = (R[1, 0] - R[0, 1]) / (2 * sin(θ))

where trace(R) is the sum of the elements on the main diagonal of R.

Common Issues and Bugs​

Different Simulator, Different Rotation Convention​

Both Euler Angles and Quaternions adhere to multiple conventions. Various software libraries utilize different conventions, which can potentially lead to errors when these libraries are used in tandem, a situation that occurs quite frequently.

For instance, some libraries represent Quaternion as $(w, x, y, z)$, positioning the real part as the first element, while others represent it as $(x, y, z, w)$. The following table illustrates the convention adopted by some widely used software and simulators.

Besides the convention of quaternion. It's essential to recognize that several popular game engines, including Unity and Unreal Engine 4, operate within a left-handed coordinate framework. Within this system, the positive x-axis extends to the right, the positive y-axis ascends upwards, and the positive z-axis stretches forward. These game engines are not only pivotal in game development but also serve as simulators in various research domains.

Conversely, the majority of simulation engines adopt a right-handed coordinate system. The distinction between left-handed and right-handed coordinate systems is a critical aspect to consider during development.

When integrating different libraries and tools, this variation in coordinate system conventions can lead to discrepancies in spatial calculations and visual representations. As such, maintaining consistency across these systems is key to ensuring accurate and reliable outcomes in your projects.

Conclusion​

Understanding 3D rotation representations and their conversion plays a pivotal role in creating sophisticated and realistic physical simulations. While this tutorial provides a comprehensive overview of the primary rotation representations, it's up to developers to determine which representation best suits their specific use-cases and computational constraints.

In this blog, we delve into the mechanics of differentiable simulators and explore why they are sometimes a more advantageous choice compared to reinforcement learning (RL) methods.

What is a Differentiable Simulator?​

Imagine a robot in an environment where, in each state $s \in \mathcal{S}$, the agent can execute an action $a \in \mathcal{A}$ leading to a subsequent state $s' \in \mathcal{S}$. We can describe this transition with the function $f: \mathcal{S} \times \mathcal{A} \to \mathcal{S}$. In conventional non-differentiable simulators, this function $f$ is often treated as a black box, with observed rewards $r \in \mathcal{R}$ serving as the primary signal for RL-based learning.

Contrastingly, in differentiable simulators, the function $f$ is perceived as an end-to-end differentiable operator. This implies that if we define some loss related to the output state $l(s')$, it becomes feasible to compute the gradient in relation to the input state and actions, such as $\frac{\partial l(s')}{\partial s}$ and $\frac{\partial l(s')}{\partial a}$. This capability enables the optimization of an entire sequence of actions using the chain rule.

Why are Differentiable Simulators Effective for Policy Learning?​

As Yann LeCun insightfully noted at NeuIPS 2016, "If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning" (source). The key takeaway here is the direct supervisory gradient flow in relation to the actions we aim to optimize, in contrast to the indirect reward-based approach of the REINFORCE algorithm in RL.

For instance, consider a task where the goal is to maneuver an object to a target position using a pusher. In an RL scenario, this would require extensive exploration, potentially involving thousands of trials before significant progress is made. Conversely, in a differentiable simulator, each iteration of gradient descent inherently contributes to progress toward the goal.

Miscellaneous Considerations​

While differentiable simulators present certain advantages over RL, they are not without their limitations and challenges, such as:

• Invalid Gradients in Certain Scenarios: Consider a scenario involving a rigid rolling pin used to flatten dough. If the initial sequence of actions fails to make contact with the dough, the resulting gradient will be zero throughout. To address this, some studies, like PlasticineLab, suggest incorporating a contact loss based on proximity to the target object. Others propose 'softening' rigid tools by increasing their influence radius, allowing objects to be affected without direct contact.

• Limited Efficiency in Long-Horizon Tasks: As discussed in this paper, the dependency of differentiable physics on local gradients poses significant challenges. The loss landscape in these scenarios is often complex and riddled with potentially misleading local optima, which can diminish the reliability of this method for certain tasks.

Robotics, as an interdisciplinary field, relies on foundational principles of physics, mathematics, control systems, and computer science to create intelligent machines. This guide will introduce the key concepts essential for robotics, covering transformation, dynamics, etc.

Rodrigues' Rotation Formula​

Rodrigues' rotation formula provides a simple and efficient way to rotate a vector in 3D around a specified axis by a given angle. Given a unit vector $\mathbf{u}$ representing the axis of rotation and an angle $\theta$, the formula for rotating a vector $\mathbf{v}$ is given by:

Here, $\mathbf{v}_{\text{rot}}$ is the rotated vector.

This formula elegantly decomposes the rotated vector into three components: a projection along the rotation axis that remains unchanged, a component perpendicular to the axis that rotates in a plane, and a cross-product term to handle rotation-induced perpendicularity.

Prove by Exponential Map​

The exponential map approach leverages the mathematical relationship between rotation matrices and the Lie algebra associated with SO(3) (the special orthogonal group in 3D).

1. Lie Algebra and Exponential Map: In this context, any rotation can be represented as an exponential of a skew-symmetric matrix formed from the axis of rotation $\mathbf{u}$. Specifically, the rotation matrix $\mathbf{R}$ for a rotation by an angle $\theta$ around the axis $\mathbf{u}$ is given by:

   $$\mathbf{R}(\theta) = \exp(\theta [\mathbf{u}]_\times)$$

   Here, $[\mathbf{u}]_\times$ denotes the skew-symmetric matrix of the unit vector $\mathbf{u}$:

   $$[\mathbf{u}]_\times = \begin{bmatrix} 0 & -u_z & u_y \\ u_z & 0 & -u_x \\ -u_y & u_x & 0 \end{bmatrix}$$

2. Exponential Series Expansion: We expand $\exp(\theta [\mathbf{u}]_\times)$ using its Taylor series:

   $\exp(\theta [\mathbf{u}]_\times) = \mathbf{I} + \sin(\theta) [\mathbf{u}]_\times + (1 - \cos(\theta)) [\mathbf{u}]_\times^2$

   where $ \mathbf{I} $ is the identity matrix. Applying this matrix to the vector $ \mathbf{v} $, we derive the components of the Rodrigues' formula:

   $$\mathbf{v}_{\text{rot}} = (\mathbf{I} + \sin(\theta) [\mathbf{u}]_\times + (1 - \cos(\theta)) [\mathbf{u}]_\times^2) \mathbf{v}$$

3. Interpretation: This expression matches the form of Rodrigues' rotation formula, breaking the rotation into linear and cross-product terms with respect to the axis $\mathbf{u}$ and angle $\theta$.

Prove by Geometry​

The geometric proof of Rodrigues' rotation formula involves decomposing the vector $\mathbf{v}$ into parallel and perpendicular components relative to the axis of rotation $\mathbf{u}$.

1. Decomposition of $\mathbf{v}$: We first decompose $\mathbf{v}$ into two components:

   • Parallel component: $\mathbf{v}_{\parallel} = (\mathbf{u} \cdot \mathbf{v}) \mathbf{u}$
   • Perpendicular component: $\mathbf{v}_{\perp} = \mathbf{v} - \mathbf{v}_{\parallel}$

2. Rotation of the Perpendicular Component: The perpendicular component $\mathbf{v}_{\perp}$ lies in the plane orthogonal to $\mathbf{u}$. When rotating $\mathbf{v}_{\perp}$ around $\mathbf{u}$ by an angle $\theta$, we obtain:

   $$\mathbf{v}_{\perp, \text{rot}} = \mathbf{v}_{\perp} \cos \theta + (\mathbf{u} \times \mathbf{v}_{\perp}) \sin \theta$$

   Since $\mathbf{v}_{\perp} = \mathbf{v} - (\mathbf{u} \cdot \mathbf{v}) \mathbf{u}$, we can substitute and simplify to get:

   $$\mathbf{v}_{\text{rot}} = \mathbf{v}_{\parallel} + \mathbf{v}_{\perp} \cos \theta + (\mathbf{u} \times \mathbf{v}) \sin \theta$$

3. Combine Components: Adding the parallel component (which remains unchanged during rotation) and the rotated perpendicular component gives the full Rodrigues' formula:

   $$\mathbf{v}_{\text{rot}} = (\mathbf{u} \cdot \mathbf{v}) \mathbf{u} + (\mathbf{v} - (\mathbf{u} \cdot \mathbf{v}) \mathbf{u}) \cos \theta + (\mathbf{u} \times \mathbf{v}) \sin \theta$$

This geometric decomposition intuitively explains how the rotation occurs in three dimensions around the axis $\mathbf{u}$ by an angle $\theta$, preserving the properties of length and orthogonality.

Forward and Inverse Kinematics​

Kinematics is a fundamental concept in robotics that deals with the motion of robot parts without considering the forces that cause the motion. Two critical types of kinematics are forward kinematics (FK) and inverse kinematics (IK), which define how robots move and interact with their environments.

Forward Kinematics (FK)​

Forward kinematics involves computing the position and orientation of a robot's end-effector (e.g., the tip of a robotic arm or gripper) given specific joint angles or displacements. The goal is to determine where the end-effector will be in the robot's workspace when the individual joints are moved in a predefined way. Forward kinematics typically uses transformation matrices, such as Denavit-Hartenberg (DH) parameters, to map joint positions to the corresponding end-effector position and orientation in 3D space.

Example:

Consider a simple two-link robotic arm with two rotational joints. Given the angles of these joints, the forward kinematics computation can determine the precise position of the end-effector relative to the robot's base. This is often represented as a chain of matrix transformations:

1. Compute Transformations for Each Joint: Each joint's rotation or translation is represented by a transformation matrix.
2. Chain the Transformations: Multiply the transformation matrices to derive the overall pose (position and orientation) of the end-effector.

Forward kinematics is typically straightforward to compute and results in a unique solution for a given set of joint positions.

Common Packages for Forward Kinematics:

• ROS (Robot Operating System): ROS offers libraries such as tf and robot_state_publisher to compute FK for robot models described in URDF (Unified Robot Description Format).

• MoveIt!: This is a powerful motion planning framework in ROS that provides FK capabilities among many other functions.

• PyBullet: This physics engine for simulating robots offers FK functions for articulated robots.

• Drake: Developed by MIT, this robotics library includes both FK and IK solvers and is particularly powerful for optimization-based approaches.

Inverse Kinematics (IK)​

Inverse kinematics works in the reverse direction: it determines the joint angles or movements needed to place the end-effector at a desired position and orientation. Unlike forward kinematics, IK can be more challenging because there may be multiple possible solutions, no solution, or constraints due to the robot's physical limits, joint boundaries, and obstacles in the environment.

Example:

Suppose you would like to move the end-effector of the same two-link robotic arm to a specific point in space. The inverse kinematics algorithm calculates the angles for each joint required to reach that position. However, depending on the arm's configuration, there could be multiple sets of angles (known as solutions) that achieve the desired end-effector position.

Challenges in Inverse Kinematics:

• Multiple Solutions: There can be more than one valid way to position the joints for a given end-effector location, especially in complex or redundant systems.
• No Solutions: Certain desired positions may be outside the robot's reachable workspace, leading to situations where no joint configuration can achieve the target.
• Constraints and Singularities: Physical constraints (like joint limits) and singularities (positions where the robot loses degrees of freedom or encounters instability) can complicate the solution process.

Inverse kinematics often requires numerical methods, such as Jacobian-based approaches, gradient descent, or optimization algorithms, to find feasible joint angles that meet a target end-effector pose. In some cases, closed-form solutions may exist, providing exact solutions without iterative calculations.

Dynamics in Robotics​

Dynamics in robotics deals with understanding the forces and torques that cause motion. Unlike kinematics, which focuses on motion without considering what causes it, dynamics considers the effect of physical quantities like mass, inertia, and external forces. Understanding dynamics is essential for accurately controlling and simulating the behavior of robots, especially when interacting with their environments.

Forward Dynamics​

Forward dynamics determines how a robot moves in response to applied forces and torques. Given a set of joint torques or forces, forward dynamics computes the resulting joint accelerations, velocities, and positions over time. This process requires solving the equations of motion that describe the robot's behavior.

Example:

Consider a robotic arm with several joints, each affected by gravity, friction, and external forces like contact with an object. Given the forces applied to the arm's joints, forward dynamics calculates how the arm will accelerate, which in turn determines how it moves through space.

Applications:

• Simulation: Forward dynamics is widely used in robotic simulators to predict how robots will move and interact with their environments based on physical forces.
• Motion Planning: By understanding how a robot will respond to forces, planners can generate physically feasible motions.
• Control: Robots can use forward dynamics to predict future states and adjust control inputs accordingly.

Inverse Dynamics​

Inverse dynamics works in the opposite direction: it computes the required forces and torques at each joint to achieve a specified motion. Given a desired trajectory of joint positions, velocities, and accelerations, inverse dynamics calculates the forces or torques necessary to produce that motion. This process is critical for controlling robots accurately and efficiently.

Example: Suppose a robotic arm needs to lift a heavy object along a predefined trajectory. The inverse dynamics approach calculates the torques that each joint motor must exert to follow the trajectory while overcoming gravity, inertia, and any applied loads.

Applications:

• Control Systems: Inverse dynamics is commonly used in controllers, such as computed torque control, to ensure that the robot follows desired paths accurately.
• Trajectory Optimization: By determining the forces and torques required to achieve a motion, engineers can optimize trajectories for efficiency or specific objectives, such as minimizing energy consumption.
• Physical Interaction: Robots interacting with their environment—like pushing, pulling, or carrying objects—rely on inverse dynamics to calculate the appropriate force exertion.

Equations of Motion​

The dynamics of a robot are typically described using the Newton-Euler equations or Lagrangian mechanics:

• Newton-Euler Formulation: This approach uses Newton's laws of motion and Euler's equations for rotational motion to describe the dynamics of each link in the robot's structure. It is particularly effective for calculating joint forces and torques in a recursive manner.

Key Equations:

Linear Motion (Newton's Second Law):

where $\mathbf{F}$ is the net force acting on the body (link). $m$ is the mass of the body. $\mathbf{a}$ is the linear acceleration of the body.

Rotational Motion (Euler's Equations):

where $\boldsymbol{\tau}$ is the net torque acting on the body, $\mathbf{I}$ is the inertia tensor of the body, $\boldsymbol{\omega}$ is the angular velocity, and $\dot{\boldsymbol{\omega}}$ is the angular acceleration.

The Newton-Euler method typically involves two passes for computation:

• Forward Recursion: Calculates the velocities and accelerations of each link, starting from the base (root) and moving outward to the end-effector.
• Backward Recursion: Computes forces and torques required at each joint, starting from the end-effector and moving back toward the base.

• Lagrangian Formulation: This method uses energy functions (kinetic and potential energy) to derive the equations of motion. The Lagrangian approach often leads to more compact expressions and is useful for analytical modeling of complex systems.

where $K$ is the kinetic energy and $P$ is the potential energy. The equations of motion are derived using the Euler-Lagrange equation:

where $q_i$ are the generalized coordinates (e.g., joint positions), $\dot{q}_i$ are the generalized velocities (time derivatives of $q_i$), and $\tau_i$ are the generalized forces or torques.

Key Equations for Energy Calculation:

• Kinetic Energy:

where $m_i$ is the mass of the $i$-th link, $\mathbf{v}_i$ is the linear velocity of the $i$-th link, $\boldsymbol{\omega}_i$ is the angular velocity of the $i$-th link, and $\mathbf{I}_i$ is the inertia tensor of the $i$-th link.

• Potential Energy:

where $g$ is the gravitational constant and $h_i$ is the height of the $i$-th link in the gravitational field.

Challenges in Dynamics​

• Nonlinearity: Robot dynamics are inherently nonlinear, particularly for systems with many degrees of freedom or under the influence of significant external forces.
• Complexity and Computation: Computing dynamics can be computationally expensive, especially for high-degree-of-freedom robots or real-time applications.
• Contact and Friction: Dynamics calculations must account for interactions with the environment, such as contact forces and friction, which can introduce discontinuities and complexity in modeling.