Initial Release by alberthli · Pull Request #6 · rai-opensource/judo

alberthli · 2025-06-04T02:24:48Z

This PR adds the private development changes to judo.

As per slack discussion with @pculbertson-bdai, we have an internal checklist of action items prior to release on pypi.

pculbertson-bdai

sure, lgtm (lol)

In general: I read through everything + think the code makes sense at a high level. Flagged some style stuff/things that aren't working (benchmark in particular seems bricked on my Mac) but overall support merging to use this version as the foundation for the repo.

When we get Simon's buy-in, let's merge and do an initial release on pypi. Then we can stand up an issue board/roadmap and start whacking bugs.

pculbertson-bdai · 2025-06-04T14:55:29Z

+The `vis_name` field specifies the name of the visualization object in the GUI backend, so it doesn't matter that much. `xyz_vis_indices` indicates which indices of the array to use for the x, y, and z coordinates of the visualization. If you set it to `None`, it will use the default values specified in `xyz_vis_defaults`.
+
+<p align="center">
+  <img src="/judo/_static/images/cylinder_push.gif" alt="cylinder_push_config" width="640">


This image doesn't show up for me - is it just my machine?

i believe this will show up on the actual docs website, but it does NOT show up locally. If there's a way you guys know how to resolve the image paths thing, then let me know, I fiddled with it for a few hours but couldn't figure it out - I could get it to show up locally but not on the published site or vice versa.

pculbertson-bdai · 2025-06-09T19:07:02Z

+    @on_event("INPUT", "controller_config")
+    def update_controller_config(self, event: dict) -> None:
+        """Callback to update controller config on receiving a new config message."""
+        self.controller_config = from_event(event, self.controller_config_cls)


Why store things in two places? Shouldn't there be one "source of truth" for these config classes?

Yes - this is a holdover from some of the legacy judo code, where there were some configs bookkept in the dora node. In reality, the clean way to handle this is to have self.controller_config point to self.controller.controller_config using a getter, and have a setter set the config in the controller attribute.

I think cleaning up this internal logic is a good candidate for v0.0.2 release.

pculbertson-bdai · 2025-06-09T19:10:27Z

+    def update_states(self, event: dict) -> None:
+        """Callback to update states on receiving a new state measurement."""
+        state_msg = from_event(event, MujocoState)
+        self.states = np.concatenate([state_msg.qpos, state_msg.qvel])


We may want to consider generalizing this to include other state data - e.g., in the official rollouts they dynamically compute what's in the full physics state.

We may also want to make a note that we only support basically Lagrangian dynamics with rigid bodies / stateless actuators atm

Agree with the sentiment, but also there are some concerns which we might want to discuss.

I think we should have a more future-proof and robust way of handling the state data. One thing that is minimal and always shared (for rigid body Lagrangian dynamics) is passing times, generalized positions, and generalized velocities. If we allowed swapping the MuJoCo rollout backend out later, then adding other fields like the mocap fields could break things.

For example, in our lab, we were considering allowing custom function definitions like ODE callables for the rollouts (this is useful for things like doing rollouts with ROMs). If there were additional fields besides the ones I described above, it might break this proposed addition.

Updated docs to include a note about simulation support.

pculbertson-bdai · 2025-06-09T19:11:45Z

+            self.step()
+            self.write_states()
+
+            # Force controller to run at fixed rate specified by model dt.


We may want to introspect here and print warnings if the inner loop time is running slower than realtime. I'm seeing behavior that looks suspiciously slower than realtime when my CPU is working hard

Agreed on this.

Many of the default tasks also use a pretty high number of default threads, so we may want to converge on some reasonable numbers and/or compute the min of os.cpu_count() and the specified threads.

added a warning.

pculbertson-bdai · 2025-06-09T19:17:11Z

@@ -0,0 +1,107 @@
+# Copyright (c) 2025 Robotics and AI Institute LLC. All rights reserved.


Running benchmark from the terminal errors with:

File "/Users/preston/judo/judo/cli.py", line 51, in main_benchmark run(cfg) File "/Users/preston/miniconda3/envs/judo/lib/python3.12/site-packages/dora_utils/launch/run.py", line 32, in run node_name = f"tmp_{node_cfg.node_id}" ^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'node_id' started dora coordinator

Hmm this doesn't error for me, also doesn't error for Zach at Caltech. @slecleach does this work for you?

Preston, If you're using conda, I think you may want to consider nuking and recreating your environment. I made some small changes to dora_utils to prep for release, so if you're using an old env, you might have some weird issues like this.

pculbertson-bdai · 2025-06-09T19:22:09Z

+        self.times = self.task.data.time + self.spline_timesteps
+        self.update_spline(self.times, self.nominal_knots)
+
+    def update_traces(self) -> None:


this will probably need a refactor for readability at some point - the shapes end up being pretty arcane. curious if we could try bringing back the CatmullRomSpline type + if the performance is any different these days

Added to the issue tracker.

Let's reevaluate the CatmullRomSpline from Viser, if'that's too slow we could stick to plotting segments.

pculbertson-bdai · 2025-06-09T19:23:51Z

+        self.traces = np.reshape(elites, (total_traces_rollouts, 2, 3))
+
+
+def make_spline(times: np.ndarray, controls: np.ndarray, spline_order: str) -> interp1d:


Having this util and the constructor in SplineData seems redundant/hard to maintain. I think we should refactor to use the class method from SplineData or vice versa.

added an issue for this.

pculbertson-bdai · 2025-06-09T19:26:05Z

+    def pre_optimization(self, old_times: np.ndarray, new_times: np.ndarray) -> None:
+        """Update sigma if the number of nodes has changed."""
+        if len(self.sigma) != self.num_nodes:
+            self.sigma = interp1d(


Probably should be refactored to a unified spline call

added an issue for this.

slecleach · 2025-06-09T21:17:19Z

Hi Albert! thanks so much for putting this PR together. This is a huge undertaking and it's addressing some critical pain points and limitations with the current version of Judo.

Some initial comments after I went through the documentation, the readme, the installation and docs building scripts.
I still need to go through the code and github actions. I'll add more comments when I am done with this.
Tested:

tested conda install
tested docs build: only issue are the instructions see below
tested judo app
tested benchmark
proof-read the documentation

Suggested fixes:

for the github actions, we should make it consistent for the python and Ubuntu versions. We were using ubuntu 22.04 and python 3.10. I am happy to move to a matrix-based test where we test both 3.10 + 22.04 and 3.13 + 24.04. We need to investigate the consequences on the dependencies though.
add badges at the top of the README, build status, docs status, codecov, documentation link, supported python versions
Docs building instructions are confusing because cd .. is implicit. One proposal could be:

cd docs
python -m venv .docs_venv [not sure we want to mention venv though]
source .docs_venv/bin/activate
pip install -r requirements.txt
cd ..
sphinx-build docs/source docs/build -b dirhtml
python -m http.server --directory docs/build 8000

Minor fixes: [we can put this on the roadmap for later development if we agree on these points]

w_pusher_velocity for the cylinder push task, the slider is stuck between 0 and 0, revamp the logic for the min_value and max value of the slider depending on the value of the default (case1 default < 0, case2 default=0, case3 default > 0) I think our logic is only good for case3 atm.
caltech_leap_cube visualization of fingers
README “For developers, to build docs locally, run the following in your environment. Note that asset paths will be broken locally that work correctly on Github Pages.” some typo maybe
Meshes not showing up. If a mesh isn’t showing up in the GUI, make sure that the geoms are named. If there are multiple mesh geoms that are not named, they will not show up in the GUI. I fixed this issue on the RAI side, I can PR this soon (just add give a unique name to each on the viser side). I don’t think we should expect people to modify their XMLs as the ones on mujoco-menagerie are often missing geometry names.
docs warning treated as errors (same as Nerfstudio); this is not a priority atm
dice icon at the bottom right of the docs pages: this is a remain from Jacta’s docs

…actly

alberthli · 2025-06-10T04:33:53Z

@slecleach

Question: how do I fix the dice in the bottom right of the docs? I can't find where the symbol is originating from. I'll happily remove it before merging this PR.

Addressed the following:

Updated code_checks.yml to use a strategy matrix testing 22.04/24.04 and python 3.10-3.13.
Added requested badges. The link to the docs already appears in the README. I don't think I have permissions to add links on the repo to things like the docs, but you should do this.
Fixed the doc building instruction typo - thanks for the catch to both Preston and Simon
caltech_leap_cube fingertips not showing up. In particular, this is unrelated to the meshes not showing up issue that you pointed out above (in fact, I independently ran into this and already have a fix for it). I have an undocumented feature that lets you filter geoms from being visualized based on whether substrings are in their names. I accidentally left the substring "collision" in the fingertip geom names in this XML. Removing them fixes this issue.

Relegated the following to the v0.0.2 release:

Make docs warnings errors
Revamping slider logic. In the meantime, I patched the logic somewhat so that you can input max/min values greater than what is specified and the limits will update to accommodate that choice, since it's a lot better than needing to restart the entire app.

initial release

f04b6eb

alberthli requested review from pculbertson-bdai and slecleach as code owners June 4, 2025 02:24

alberthli added 2 commits June 3, 2025 21:17

minor additions to README

c965cf5

adjust num_rollouts down on fr3_pick defaults

35bab60

pculbertson-bdai approved these changes Jun 9, 2025

View reviewed changes

alberthli added 7 commits June 9, 2025 20:22

address docs nit

d643247

update docs to provide more info about what the simulator supports ex…

3686b4d

…actly

add a warning when simulation step is slow

c782541

make warning shorter

ce16b03

added a matrix for ubuntu and python versions

3d45755

add badges

37b815f

fix caltech leap cube fingertips not showing

0d8ed01

alberthli added 9 commits June 10, 2025 10:40

update emoji on docs

2a16cd3

change to v0.0.1

2d90d4c

updated badges to have all python versions

d029b32

updated python versions in pyproject.toml

5b32ea7

update description

113b870

update lock file

9851433

deploy docs only on push to main

5edc121

separate code checks into precommit and test workflows

9dc9996

fix gui element removal bug

d52ab84

slecleach merged commit 2189498 into rai-opensource:main Jun 10, 2025
17 checks passed

alberthli deleted the alberthli/initial-release branch June 10, 2025 19:20

		@@ -0,0 +1,107 @@
		# Copyright (c) 2025 Robotics and AI Institute LLC. All rights reserved.

		self.traces = np.reshape(elites, (total_traces_rollouts, 2, 3))


		def make_spline(times: np.ndarray, controls: np.ndarray, spline_order: str) -> interp1d:

Conversation

alberthli commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pculbertson-bdai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slecleach commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alberthli commented Jun 10, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alberthli commented Jun 4, 2025 •

edited

Loading

slecleach commented Jun 9, 2025 •

edited

Loading