-
Notifications
You must be signed in to change notification settings - Fork 24
Expand file tree
/
Copy pathindex.html
More file actions
1713 lines (1492 loc) · 111 KB
/
index.html
File metadata and controls
1713 lines (1492 loc) · 111 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE HTML>
<html>
<head>
<!-- Google analytics tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-STGLQW4BJX"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-STGLQW4BJX');
</script>
<!-- Title -->
<title>Andy Zeng - Generalist</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=1000">
<!-- Isotope JS -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.6.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.13.2/jquery-ui.min.js"></script>
<script src="https://unpkg.com/isotope-layout@3/dist/isotope.pkgd.min.js"></script>
<!-- Custom Style -->
<link rel="stylesheet" href="style.css">
<!-- Google Font -->
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Asap:ital,wght@0,100;0,200;0,300;0,400;0,500;0,600;0,700;1,100;1,200;1,300;1,400;1,500;1,600;1,700&display=swap" rel="stylesheet">
<style>
@import url('https://fonts.googleapis.com/css2?family=Asap:ital,wght@0,100;0,200;0,300;0,400;0,500;0,600;0,700;1,100;1,200;1,300;1,400;1,500;1,600;1,700&display=swap');
</style>
</head>
<body id="body">
<div id="main">
<div id="intro">
<div id="intro-text">
<h1>Andy Zeng</h1>
<p>
Andy is a co-founder at <a href="https://generalistai.com/">Generalist</a>. Before that, he was a research scientist and tech lead at Google DeepMind – and before that, a student at Princeton and UC Berkeley. Andy has worked on training <a href="https://research.google/blog/palm-e-an-embodied-multimodal-language-model/">large-scale foundation models</a>, developing <a href="https://research.google/blog/robots-that-write-their-own-code/">robots that write their own code</a>, and inventing <a href="https://graspinwild.cs.columbia.edu/">handheld data collection for robots</a>. These days, he is interested in training large neural nets on massive amounts of robot data.
<div id="more-bio" style="display: None">
<br>
<p>Andy Zeng is a co-founder at <a href="https://generalistai.com/">Generalist</a>, where he leads a small team working on self-improving Foundation models for robots. He received his Bachelors in Computer Science and Mathematics at <a href="https://www.berkeley.edu/">UC Berkeley</a>, and his PhD in Computer Science at <a href="https://www.princeton.edu/">Princeton</a>. He is interested in building algorithms that enable machines to intelligently interact with the world and improve themselves over time. Andy received Best Paper Awards from HRI '24, CoRL '23, ICRA '23, T-RO '20, RSS'19, and has been finalist for paper awards at RSS '23, CoRL '20 - '22, ICRA '20, RSS '19, IROS '18. He led machine learning as part of Team MIT-Princeton, winning 1st place (stow task) at the worldwide Amazon Picking Challenge '17. Andy is a recipient of the Princeton SEAS Award for Excellence, Japan Foundation Paper Award, NVIDIA Fellowship, and Gordon Y.S. Wu Fellowship in Engineering and Wu Prize. His work has been featured in the press, including the New York Times, BBC, and Wired.</p>
</div>
<br>
<!-- <a href="javascript:toggle_bio()">Formal Bio</a> -->
<a href="https://twitter.com/andyzengineer">X (Twitter)</a>
<a href="https://scholar.google.com/citations?user=q7nFtUcAAAAJ&hl=en">G. Scholar</a>
<a href="https://www.linkedin.com/in/andyzengineer">LinkedIn</a>
<a href="https://github.com/andyzeng">Github</a>
<br><br>
andyzeng at generalistai dot com
<br><br>
</p>
</div>
<div id="intro-image">
<img src="images/profile.jpg">
</div>
</div>
<div id="filters" class="button-group">
<!-- <button class="button" data-filter="*">Show All</button> -->
<button class="button" data-filter=".highlight">Highlights</button>
<button class="button is-checked" data-filter=".publication">Publications</button>
<button class="button" data-filter=".talk">Talks</button>
<button class="button" data-filter=".misc">Misc</button>
</div>
<div class="grid">
<!-- Preview Videos -->
<div class="list-item highlight previews" data-category="highlight">
<a href="https://generalistai.com/"><video class="preview1" playsinline="" muted="" autoplay="" loop=""><source src="images/video-generalist-teaser-crop.mp4" type="video/mp4"></video></a>
<a href="https://say-can.github.io/"><video class="preview2" playsinline="" muted="" autoplay="" loop=""><source src="images/video-saycan.mp4" type="video/mp4"></video></a>
<a href="https://tossingbot.cs.princeton.edu/"><video class="preview3" playsinline="" muted="" autoplay="" loop=""><source src="images/video-tossingbot-crop.mp4" type="video/mp4"></video></a>
</div>
<!-- Truncated Set of Highlights (Shown by Default) -->
<div id="main-highlights">
<div class="list-item highlight" data-category="highlight">
<p class="date">2026</p>Generalist article <a href="https://generalistai.com/blog/apr-02-2026-GEN-1">"GEN-1: Scaling Embodied Foundation Models to Mastery"</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date"></p>Generalist blog post <a href="https://generalistai.com/blog/jan-29-2026-physical-commonsense">"The Dark Matter of Robotics: Physical Commonsense"</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date">2025</p>Generalist article <a href="https://generalistai.com/blog/nov-04-2025-GEN-0">"GEN-0: Embodied Foundation Models That Scale with Physical Interaction"</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date"></p>Out of stealth! Co-founded <a href="https://generalistai.com/">Generalist</a> to build general intelligence for the physical world
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date">2024</p>IEEE Robotics and Automation (RAS) <a href="https://www.ieee-ras.org/awards-recognition/society-awards/ras-early-career-award-industry-or-government">Early Career Award</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date">2023</p>Conference on Robot Learning (CoRL) <a href="https://robot-help.github.io/">Best Student Paper Award</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date"></p>IEEE International Conference on Robotics and Automation (ICRA) <a href="https://code-as-policies.github.io/">Outstanding Learning Paper Award</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date">2022</p>Google AI blog post <a href="https://ai.googleblog.com/2022/11/robots-that-write-their-own-code.html">"Robots That Write Their Own Code"</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date"></p>AXIOS article <a href="https://www.axios.com/2022/11/03/google-artificial-intelligence">"Unleash All This Creativity: Google AI's Breathtaking Potential"</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date"></p>CNET video <a href="https://www.youtube.com/watch?v=dCPHGwW9SOk">"Google’s Most Advanced Robot Brain Just Got a Body"</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date">2020</p>IEEE Transactions on Robotics (T-RO) <a href="https://www.ieee-ras.org/publications/t-ro">Best Paper Award</a>
</div>
<div class="list-item highlight" data-category="highlight">
<p class="date">2019</p>New York Times article <a href="http://andyzeng.github.io/images/nytimes-business-newspaper-tossingbot.png">"A New Lab Full of Fast Learners"</a>
</div>
</div>
<!-- Publications -->
<div class="list-item publication description" data-category="publication">
For a more up-to-date list, visit my <a href="https://scholar.google.com/citations?user=q7nFtUcAAAAJ&hl=en">Google Scholar</a>
</div>
<div id="main-publications">
<div class="list-item publication" data-category="publication">
<a href="https://palm-e.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/video-palm-e.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://palm-e.github.io/">PaLM-E: An Embodied Multimodal Language Model</a></h3>
<p>
Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, Pete Florence<br>
<i>International Conference on Machine Learning (ICML) 2023</i><br>
<a href="https://palm-e.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2303.03378">PDF</a> •
<a href="https://palm-e.github.io/#demo">Demo</a> •
<a href="https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal-language.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://implicitbc.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/teaser-ibc.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://implicitbc.github.io/">Implicit Behavioral Cloning</a></h3>
<p>
Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, Jonathan Tompson<br>
<i>Conference on Robot Learning (CoRL) 2021</i><br>
<a href="https://implicitbc.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2109.00137.pdf">PDF</a> •
<a href="https://github.com/google-research/ibc">Code</a> •
<a href="https://ai.googleblog.com/2021/11/decisiveness-in-imitation-learning-for.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://video-language-planning.github.io/" class="thumbnail">
<img src="https://video-language-planning.github.io/img/share_image.png" alt="" />
</a>
<div class="project-description">
<h3><a href="https://video-language-planning.github.io/">Video Language Planning</a></h3>
<p>
Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B. Tenenbaum, Leslie Kaelbling, Andy Zeng, Jonathan<br> Tompson<br>
<i>International Conference on Learning Representations (ICLR) 2024</i><br>
<a href="https://video-language-planning.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2310.10625">PDF</a> •
<a href="https://github.com/video-language-planning/vlp_code">Code</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://code-as-policies.github.io/" class="thumbnail"><img src="images/code-as-policies-teaser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://code-as-policies.github.io/">Code as Policies: Language Model Programs for Embodied Control</a></h3>
<p>
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, Andy Zeng<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2023</i><br>
<font color="49bf9"><i>★ Outstanding Learning Paper Award, ICRA ★</i></font><br>
<a href="https://code-as-policies.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2209.07753.pdf">PDF</a> •
<a href="https://github.com/google-research/google-research/tree/master/code_as_policies">Code</a> •
<a href="https://colab.research.google.com/drive/1V9GU70GQN-Km4qsxYqvR-c0Sgzod19-j">Colab</a> •
<a href="https://ai.googleblog.com/2022/11/robots-that-write-their-own-code.html">Google AI Blog</a> •
<a href="https://techcrunch.com/2022/11/02/google-wants-robots-to-generate-their-own-code/">TechCrunch</a> •
<a href="https://www.axios.com/2022/11/03/google-artificial-intelligence">AXIOS</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/pdf/2203.08715.pdf" class="thumbnail"><img src="images/infuser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://arxiv.org/pdf/2203.08715.pdf">Multiscale Sensor Fusion and Continuous Control with Neural CDEs</a></h3>
<p>
Sumeet Singh, Francis McCann Ramirez, Jacob Varley, Andy Zeng, Vikas Sindhwani<br>
<i>IEEE International Conference on Intelligent Robots and Systems (IROS) 2022</i><br>
<a href="https://arxiv.org/pdf/2203.08715.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://robot-help.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/knowno-teaser.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://robot-help.github.io/">Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners</a></h3>
<p>
Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu, Dorsa Sadigh, Andy Zeng, Anirudha Majumdar<br>
<i>Conference on Robot Learning (CoRL) 2023</i><br>
<font color="49bf9"><i>★ Oral Presentation, Best Student Paper Award, CoRL ★</i></font><br>
<a href="https://robot-help.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2307.01928">PDF</a> •
<a href="https://youtu.be/xCXx09gfhx4">Video</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/pdf/2110.04367.pdf" class="thumbnail"><img src="images/hybrid-random-features.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://arxiv.org/pdf/2110.04367.pdf">Hybrid Random Features</a></h3>
<p>
Krzysztof Choromanski, Haoxian Chen, Han Lin, Yuanzhe Ma, Arijit Sehanobish, Deepali Jain, Michael S Ryoo, Jake Varley, Andy Zeng, Valerii Likhosherstov, Dmitry Kalashnikov, Vikas Sindhwani, Adrian Weller<br>
<i>The International Conference on Learning Representations (ICLR) 2022</i><br>
<a href="https://arxiv.org/pdf/2110.04367.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://x-irl.github.io/" class="thumbnail"><img src="images/xirl-overlay.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://x-irl.github.io/">XIRL: Cross-Embodiment Inverse Reinforcement Learning</a></h3>
<p>
Kevin Zakka, Andy Zeng, Pete Florence, Jonathan Tompson, Jeannette Bohg, Debidatta Dwibedi<br>
<i>Conference on Robot Learning (CoRL) 2021</i><br>
<font color="49bf9"><i>★ Best Paper Award Finalist, CoRL ★</i></font><br>
<a href="https://x-irl.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2106.03911.pdf">PDF</a> •
<a href="https://github.com/google-research/google-research/tree/master/xirl">Code</a> •
<a href="https://github.com/kevinzakka/x-magical">Benchmark</a> •
<a href="https://ai.googleblog.com/2022/02/robot-see-robot-do.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://graspinwild.cs.columbia.edu/" class="thumbnail"><img src="images/teaser-grasp-in-wild.jpg" alt="" /></a>
<div class="project-description">
<h3><a href="https://graspinwild.cs.columbia.edu/">Grasping in the Wild: Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations</a></h3>
<p>
Shuran Song, Andy Zeng, Johnny Lee, Thomas Funkhouser<br>
<i>IEEE International Conference on Intelligent Robots and Systems (IROS) 2020</i><br>
<i>IEEE Robotics and Automation Letters (RA-L) 2020</i><br>
<a href="https://graspinwild.cs.columbia.edu/">Webpage</a> •
<a href="https://arxiv.org/pdf/1912.04344.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="http://tossingbot.cs.princeton.edu/" class="thumbnail"><img src="images/teaser-tossingbot.jpg" alt="" /></a>
<div class="project-description">
<h3><a href="https://tossingbot.cs.princeton.edu/">TossingBot: Learning to Throw Arbitrary Objects with Residual Physics</a></h3>
<p>
Andy Zeng, Shuran Song, Johnny Lee, Alberto Rodriguez, Thomas Funkhouser<br>
<i>Robotics: Science and Systems (RSS) 2019</i><br>
<i>IEEE Transactions on Robotics (T-RO) 2020</i><br>
<font color="49bf9"><i>★ Best Paper Award, T-RO & Best Systems Paper Award, RSS ★</i></font><br>
<a href="https://tossingbot.cs.princeton.edu/">Webpage</a> •
<a href="https://arxiv.org/pdf/1903.11239.pdf">PDF</a> •
<!-- <a href="https://github.com/andyzeng/visual-pushing-grasping">Code</a> • -->
<a href="https://ai.googleblog.com/2019/03/unifying-physics-and-deep-learning-with.html">Google AI Blog</a> •
<a href="images/nytimes-business-newspaper-tossingbot.png">New York Times</a> •
<a href="https://spectrum.ieee.org/automaton/robotics/artificial-intelligence/google-teaches-robot-to-toss-bananas-better-than-you-do">IEEE Spectrum</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="http://arc.cs.princeton.edu" class="thumbnail"><img src="images/teaser-amazon-robotics-challenge.jpg" alt="" /></a>
<div class="project-description">
<h3><a href="http://arc.cs.princeton.edu">Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching</a></h3>
<p>
Andy Zeng, Shuran Song, Kuan-Ting Yu, Elliott Donlon, Francois R. Hogan, Maria Bauza, Daolin Ma, Orion Taylor, Melody Liu, Eudald Romo, Nima Fazeli, Ferran Alet, Nikhil Chavan Dafle, Rachel Holladay, Isabella Morona, Prem Qu Nair, Druck Green, Ian Taylor, Weber Liu, Thomas Funkhouser, Alberto Rodriguez<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2018</i><br>
<i>The International Journal of Robotics Research (IJRR) 2019</i><br>
<font color="49bf9"><i>★ 1st Place (Stow Task), Amazon Robotics Challenge 2017 ★</i></font><br>
<a href="http://arc.cs.princeton.edu">Webpage</a> •
<a href="https://arxiv.org/pdf/1710.01330.pdf">PDF</a> •
<a href="https://github.com/andyzeng/arc-robot-vision">Code</a> •
<a href="http://news.mit.edu/2018/robo-picker-grasps-and-packs-0220">MIT News</a> •
<!-- <a href="https://www.youtube.com/watch?v=yVIRLao1E28">Amazon News</a> • -->
<a href="https://www.engadget.com/2018/02/20/robot-pick-up-sort-object-improve-warehouse-efficiency/">Engadget</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="http://3dmatch.cs.princeton.edu/" class="thumbnail"><img src="images/teaser-3dmatch.jpg" alt="" /></a>
<div class="project-description">
<h3><a href="http://3dmatch.cs.princeton.edu/">3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions</a></h3>
<p>
Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, Thomas Funkhouser
<br>
<i>IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017</i><br>
<font color="49bf9"><i>★ Oral Presentation, CVPR ★</i></font><br>
<a href="http://3dmatch.cs.princeton.edu/">Webpage</a> •
<a href="https://arxiv.org/pdf/1603.08182.pdf">PDF</a> •
<a href="https://github.com/andyzeng/3dmatch-toolbox">Code</a> •
<a href="https://www.youtube.com/watch?v=qNVZl7bCjsU&list=PL_bDvITUYucADb15njRd7geem8vxOyo6N&index=3">Talk</a> •
<a href="https://www.youtube.com/watch?v=1U3YKnuMS7g">2 Minute Papers</a>
</p>
</div>
</div>
</div>
<!-- All Publications (Click to Show) -->
<div id="more-publications" style="display: None">
<div class="list-item publication" data-category="publication">
<a href="https://robot-teaching.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/daynight.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://robot-teaching.github.io/">Learning to Learn Faster from Human Feedback with Language Model Predictive Control</a></h3>
<p>
Jacky Liang, Fei Xia, Wenhao Yu, Andy Zeng, Montserrat Gonzalez Arenas, Maria Attarian, Maria Bauza, Matthew Bennice, Alex Bewley, Adil Dostmohamed, Chuyuan Kelly Fu, Nimrod Gileadi, Marissa Giustina, Keerthana Gopalakrishnan, Leonard Hasenclever, Jan Humplik, Jasmine Hsu, Nikhil Joshi, Ben Jyenis, Chase Kew, Sean Kirmani, Tsang-Wei Edward Lee, Kuang-Huei Lee, Assaf Hurwitz Michaely, Joss Moore, Ken Oslund, Dushyant Rao, Allen Ren, Baruch Tabanpour, Quan Vuong, Ayzaan Wahid, Ted Xiao, Ying Xu, Vincent Zhuang, Peng Xu, Erik Frey, Ken Caluwaerts, Tingnan Zhang, Brian Ichter, Jonathan Tompson, Leila Takayama, Vincent Vanhoucke, Izhak Shafran, Maja Mataric, Dorsa Sadigh, Nicolas Heess, Kanishka Rao, Nik Stewart, Jie Tan, Carolina Parada<br>
<i>Robotics: Science and Systems (RSS) 2024</i><br>
<a href="https://robot-teaching.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2402.11450">PDF</a> •
<a href="https://colab.research.google.com/drive/1YcRN_kklw3cVVJNvgK_IEV6nDce9EJWK?usp=sharing">Code</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/abs/2402.05741" class="thumbnail">
<img src="images/fm-review.jpeg" alt="" />
</a>
<div class="project-description">
<h3><a href="https://arxiv.org/abs/2402.05741">Real-World Robot Applications of Foundation Models: A Review</a></h3>
<p>
Kento Kawaharazuka, Tatsuya Matsushima, Andrew Gambardella, Jiaxian Guo, Chris Paxton, Andy Zeng<br>
<i>Advanced Robotics (AR) 2024</i><br>
<!-- <a href="https://video-language-planning.github.io/">Webpage</a> • -->
<a href="https://arxiv.org/abs/2402.05741">PDF</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://generative-expressive-motion.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/genem.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://generative-expressive-motion.github.io/">Generative Expressive Robot Behaviors using Large Language Models</a></h3>
<p>
Karthik Mahadevan, Jonathan Chien, Noah Brown, Zhuo Xu, Carolina Parada, Fei Xia, Andy Zeng, Leila Takayama, Dorsa Sadigh<br>
<i>ACM/IEEE International Conference on Human Robot Interaction (HRI) 2024</i><br>
<font color="49bf9"><i>★ Best Paper Award, HRI ★</i></font><br>
<a href="https://generative-expressive-motion.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2401.14673">PDF</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://chain-of-code.github.io/" class="thumbnail">
<img src="images/chain-of-code.png" alt="" />
</a>
<div class="project-description">
<h3><a href="https://chain-of-code.github.io/">Chain of Code: Reasoning with a Language Model-Augmented Code Emulator</a></h3>
<p>
Chengshu Li, Jacky Liang, Andy Zeng, Xinyun Chen, Karol Hausman, Dorsa Sadigh, Sergey Levine, Li Fei-Fei, Fei Xia, Brian Ichter<br>
<i>International Conference on Machine Learning (ICML) 2024</i><br>
<font color="49bf9"><i>★ Oral Presentation, ICML ★</i></font><br>
<a href="https://chain-of-code.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2312.04474">PDF</a> •
<a href="http://colab.research.google.com/github/google-research/google-research/blob/master/code_as_policies/coc_demo.ipynb">Code</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://video-language-planning.github.io/" class="thumbnail">
<img src="https://video-language-planning.github.io/img/share_image.png" alt="" />
</a>
<div class="project-description">
<h3><a href="https://video-language-planning.github.io/">Video Language Planning</a></h3>
<p>
Yilun Du, Mengjiao Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B. Tenenbaum, Leslie Kaelbling, Andy Zeng, Jonathan Tompson<br>
<i>International Conference on Learning Representations (ICLR) 2024</i><br>
<a href="https://video-language-planning.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2310.10625">PDF</a> •
<a href="https://github.com/video-language-planning/vlp_code">Code</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://general-pattern-machines.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/in-context-cartpole-teaser.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://general-pattern-machines.github.io/">Large Language Models as General Pattern Machines</a></h3>
<p>
Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng<br>
<i>Conference on Robot Learning (CoRL) 2023</i><br>
<a href="https://general-pattern-machines.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2307.04721">PDF</a> •
<a href="https://general-pattern-machines.github.io/#colabs">Code</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://robot-help.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/knowno-teaser.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://robot-help.github.io/">Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners</a></h3>
<p>
Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu, Dorsa Sadigh, Andy Zeng, Anirudha Majumdar<br>
<i>Conference on Robot Learning (CoRL) 2023</i><br>
<font color="49bf9"><i>★ Oral Presentation, Best Student Paper Award, CoRL ★</i></font><br>
<a href="https://robot-help.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2307.01928">PDF</a> •
<a href="https://youtu.be/xCXx09gfhx4">Video</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://language-to-reward.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/language-to-rewards-teaser.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://language-to-reward.github.io/">Language to Rewards for Robotic Skill Synthesis</a></h3>
<p>
Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, Fei Xia<br>
<i>Conference on Robot Learning (CoRL) 2023</i><br>
<font color="49bf9"><i>★ Oral Presentation, CoRL ★</i></font><br>
<a href="https://language-to-reward.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2306.08647">PDF</a> •
<a href="https://youtu.be/7KiKg0rdSSQ">Video</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://general-part-assembly.github.io/" class="thumbnail">
<img src="images/gpat-teaser.jpeg" alt="" />
</a>
<div class="project-description">
<h3><a href="https://general-part-assembly.github.io/">Rearrangement Planning for General Part Assembly</a></h3>
<p>
Yulong Li, Andy Zeng, Shuran Song<br>
<i>Conference on Robot Learning (CoRL) 2023</i><br>
<font color="49bf9"><i>★ Oral Presentation, CoRL ★</i></font><br>
<a href="https://general-part-assembly.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2307.00206">PDF</a> •
<a href="https://general-part-assembly.github.io/figs/video.mp4">Video</a> •
<a href="https://github.com/columbia-ai-robotics/gpat/">Code</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://openreview.net/forum?id=Nii0_rRJwN" class="thumbnail">
<img src="images/calamari-teaser.png" alt="" />
</a>
<div class="project-description">
<h3><a href="https://openreview.net/forum?id=Nii0_rRJwN">CALAMARI: Contact-Aware and Language conditioned spatial Action MApping for contact-RIch manipulation</a></h3>
<p>
Youngsun Wi, Mark Van der Merwe, Pete Florence, Andy Zeng, Nima Fazeli<br>
<i>Conference on Robot Learning (CoRL) 2023</i><br>
<a href="https://openreview.net/pdf?id=Nii0_rRJwN">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://tidybot.cs.princeton.edu/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/tidybot-teaser.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://tidybot.cs.princeton.edu/">TidyBot: Personalized Robot Assistance with Large Language Models</a></h3>
<p>
Jimmy Wu, Rika Antonova, Adam Kan, Marion Lepert, Andy Zeng, Shuran Song, Jeannette Bohg, Szymon Rusinkiewicz, Thomas Funkhouser<br>
<i>IEEE International Conference on Intelligent Robots and Systems (IROS) 2023</i><br>
<i>Autonomous Robots 2023</i><br>
<a href="https://tidybot.cs.princeton.edu/">Webpage</a> •
<a href="https://arxiv.org/pdf/2305.05658.pdf">PDF</a> •
<a href="https://github.com/jimmyyhwu/tidybot">Code</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/abs/2306.05392" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/codevqa-teaser.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://arxiv.org/abs/2306.05392">Modular Visual Question Answering via Code Generation</a></h3>
<p>
Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang, Arsha Nagrani, Cordelia Schmid, Andy Zeng, Trevor Darrell, Dan Klein<br>
<i>Association for Computational Linguistics (ACL) 2023</i><br>
<a href="https://arxiv.org/pdf/2306.05392.pdf">PDF</a> •
<a href="https://github.com/sanjayss34/codevqa">Code</a> •
<a href="https://ai.googleblog.com/2023/07/modular-visual-question-answering-via.html">Google AI Blog</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://palm-e.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/video-palm-e.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://palm-e.github.io/">PaLM-E: An Embodied Multimodal Language Model</a></h3>
<p>
Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, Pete Florence<br>
<i>International Conference on Machine Learning (ICML) 2023</i><br>
<a href="https://palm-e.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2303.03378">PDF</a> •
<a href="https://palm-e.github.io/#demo">Demo</a> •
<a href="https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal-language.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://kzakka.com/robopianist/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/robopianist.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://kzakka.com/robopianist/">RoboPianist: A Benchmark for High-Dimensional Robot Control</a></h3>
<p>
Kevin Zakka, Laura Smith, Nimrod Gileadi, Taylor Howell, Xue Bin Peng, Sumeet Singh, Yuval Tassa, Pete Florence, Andy Zeng, Pieter Abbeel<br>
<i>Conference on Robot Learning (CoRL) 2023</i><br>
<a href="https://kzakka.com/robopianist/">Webpage</a> •
<a href="https://kzakka.com/robopianist/robopianist.pdf">PDF</a> •
<a href="https://github.com/google-research/robopianist">Code</a> •
<a href="https://colab.research.google.com/github/google-research/robopianist/blob/main/tutorial.ipynb">Colab</a> •
<a href="https://kevinzakka.github.io/robopianist-demo/">Demo</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://avlmaps.github.io/" class="thumbnail"><img src="https://avlmaps.github.io/static/images/cover_lady.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://avlmaps.github.io/">Audio Visual Language Maps for Robot Navigation</a></h3>
<p>
Chenguang Huang, Oier Mees, Andy Zeng, Wolfram Burgard<br>
<i>International Symposium on Experimental Robotics (ISER) 2023</i><br>
<a href="https://avlmaps.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2303.07522">PDF</a> •
<a href="https://colab.research.google.com/drive/1gdtLvg_Fbl16N3ITp5FsU9ZAG6HmspVb?usp=sharing">Colab</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://grounded-decoding.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="https://grounded-decoding.github.io/media/beam.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://grounded-decoding.github.io/">Grounded Decoding: Guiding Text Generation with Grounded Models for Robot Control</a></h3>
<p>
Wenlong Huang, Fei Xia, Dhruv Shah, Danny Driess, Andy Zeng, Yao Lu, Pete Florence, Igor Mordatch, Sergey Levine, Karol Hausman, Brian Ichter<br>
<i>Conference on Neural Information Processing Systems (NeurIPS) 2023</i><br>
<a href="https://grounded-decoding.github.io/">Webpage</a> •
<a href="https://grounded-decoding.github.io/paper.pdf">PDF</a> •
<a href="https://www.youtube.com/watch?v=KHhAlBIQftQ">Video</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://code-as-policies.github.io/" class="thumbnail"><img src="images/code-as-policies-teaser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://code-as-policies.github.io/">Code as Policies: Language Model Programs for Embodied Control</a></h3>
<p>
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, Andy Zeng<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2023</i><br>
<font color="49bf9"><i>★ Outstanding Learning Paper Award, ICRA ★</i></font><br>
<a href="https://code-as-policies.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2209.07753.pdf">PDF</a> •
<a href="https://github.com/google-research/google-research/tree/master/code_as_policies">Code</a> •
<a href="https://colab.research.google.com/drive/1V9GU70GQN-Km4qsxYqvR-c0Sgzod19-j">Colab</a> •
<a href="https://ai.googleblog.com/2022/11/robots-that-write-their-own-code.html">Google AI Blog</a> •
<a href="https://techcrunch.com/2022/11/02/google-wants-robots-to-generate-their-own-code/">TechCrunch</a> •
<a href="https://www.axios.com/2022/11/03/google-artificial-intelligence">AXIOS</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://vlmaps.github.io/" class="thumbnail"><img src="images/vlmaps-teaser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://vlmaps.github.io/">Visual Language Maps for Robot Navigation</a></h3>
<p>
Chenguang Huang, Oier Mees, Andy Zeng, Wolfram Burgard<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2023</i><br>
<a href="https://vlmaps.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2210.05714.pdf">PDF</a> •
<a href="https://github.com/Tom-Huang/vlmaps">Code</a> •
<a href="https://ai.googleblog.com/2023/03/visual-language-maps-for-robot.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://socraticmodels.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<!-- <source src="https://socraticmodels.github.io/images/sm_2_ego_qa.mp4" type="video/mp4"> -->
<source src="images/socratic-robots.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://socraticmodels.github.io/">Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language</a></h3>
<p>
Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, Pete Florence<br>
<i>International Conference on Learning Representations (ICLR) 2023</i><br>
<font color="49bf9"><i>★ Oral Presentation, ICLR ★</i></font><br>
<a href="https://socraticmodels.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2204.00598.pdf">PDF</a> •
<a href="https://socraticmodels.github.io/#code">Code</a>
<br> <!-- TODO: extra spaces until I figure out the margin bug. -->
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://yenchenlin.me/mira/" class="thumbnail"><img src="images/mira-teaser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://yenchenlin.me/mira/">MIRA: Mental Imagery for Robotic Affordances</a></h3>
<p>
Lin Yen-Chen, Pete Florence, Andy Zeng, Jonathan T. Barron, Yilun Du, Wei-Chiu Ma, Anthony Simeonov, Alberto Rodriguez Garcia, Phillip Isola<br>
<i>Conference on Robot Learning (CoRL) 2022</i><br>
<a href="https://yenchenlin.me/mira/">Webpage</a> •
<a href="https://arxiv.org/pdf/2212.06088.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://innermonologue.github.io/" class="thumbnail"><img src="images/innermonologue-teaser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://innermonologue.github.io/">Inner Monologue: Embodied Reasoning through Planning with Language Models</a></h3>
<p>
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, Sergey Levine, Karol Hausman, Brian Ichter<br>
<i>Conference on Robot Learning (CoRL) 2022</i><br>
<a href="https://innermonologue.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2207.05608.pdf">PDF</a> •
<a href="https://www.youtube.com/watch?v=Ybk8hxKeMYQ">2 Minute Papers</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://www.mmintlab.com/research/virdopp/" class="thumbnail"><img src="images/virdoplus-teaser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://www.mmintlab.com/research/virdopp/">VIRDO++: Real-World, Visuo-Tactile Dynamics and Perception of Deformable Objects</a></h3>
<p>
Youngsun Wi, Andy Zeng, Pete Florence, Nima Fazeli<br>
<i>Conference on Robot Learning (CoRL) 2022</i><br>
<a href="https://www.mmintlab.com/research/virdopp/">Webpage</a> •
<a href="https://arxiv.org/pdf/2210.03701.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://say-can.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/saycan.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://say-can.github.io/">Do As I Can, Not As I Say: Grounding Language in Robotic Affordances</a></h3>
<p>
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Kuang-Huei Lee, Sergey Levine, Yao Lu, Linda Luu, Carolina Parada, Peter Pastor, Jornell Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego Reyes, Pierre Sermanet, Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Mengyuan Yan, Andy Zeng<br>
<i>Conference on Robot Learning (CoRL) 2022</i><br>
<font color="49bf9"><i>★ Oral Presentation, Special Innovation Award, CoRL ★</i></font><br>
<a href="https://say-can.github.io/">Webpage</a> •
<a href="https://arxiv.org/abs/2204.01691">PDF</a> •
<a href="https://github.com/google-research/google-research/tree/master/saycan">Code</a> •
<a href="https://ai.googleblog.com/2022/08/towards-helpful-robots-grounding.html">Google AI Blog</a> •
<a href="https://www.wired.com/story/google-robot-learned-to-take-orders-by-scraping-the-web/">Wired</a> •
<a href="https://www.washingtonpost.com/video/technology/google-is-training-robots-to-perform-complex-tasks/2022/08/16/3339cdbb-344b-482f-8671-33022725df81_video.html">Washington Post</a> •
<a href="https://www.youtube.com/watch?v=dCPHGwW9SOk">CNET</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://sites.google.com/berkeley.edu/cloudfolding" class="thumbnail"><img src="images/cloudfolding.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://sites.google.com/berkeley.edu/cloudfolding">Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Research</a></h3>
<p>
Ryan Hoque, Kaushik Shivakumar, Shrey Aeron, Gabriel Deza, Aditya Ganapathi, Adrian Wong, Johnny Lee, Andy Zeng, Vincent Vanhoucke, Ken Goldberg<br>
<i>IEEE International Conference on Intelligent Robots and Systems (IROS) 2022</i><br>
<a href="https://sites.google.com/berkeley.edu/cloudfolding">Webpage</a> •
<a href="https://arxiv.org/pdf/2204.10297.pdf">PDF</a> •
<a href="https://github.com/google-research/pyreach">Code</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://ieeexplore.ieee.org/abstract/document/9893496" class="thumbnail"><img src="images/multi-obj-teaser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://ieeexplore.ieee.org/abstract/document/9893496">Algorithms and Systems for Manipulating Multiple Objects</a></h3>
<p>
Zherong Pan, Andy Zeng, Yunzhu Li, Jingjin Yu, Kris Hauser<br>
<i>IEEE Transactions on Robotics (T-RO) 2022</i><br>
<a href="https://ieeexplore.ieee.org/abstract/document/9893496">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://learning-dynamic-manipulation.cs.princeton.edu" class="thumbnail"><img src="images/air-wu.gif" alt="" /></a>
<div class="project-description">
<h3><a href="https://learning-dynamic-manipulation.cs.princeton.edu">Learning Pneumatic Non-Prehensile Manipulation with a Mobile Blower</a></h3>
<p>
Jimmy Wu, Xingyuan Sun, Andy Zeng, Shuran Song, Szymon Rusinkiewicz, Thomas Funkhouser<br>
<i>IEEE International Conference on Intelligent Robots and Systems (IROS) 2022</i><br>
<i>IEEE Robotics and Automation Letters (RA-L) 2022</i><br>
<a href="https://learning-dynamic-manipulation.cs.princeton.edu">Webpage</a> •
<a href="https://arxiv.org/pdf/2204.02390.pdf">PDF</a> •
<a href="https://github.com/jimmyyhwu/learning-dynamic-manipulation">Code</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/pdf/2203.08715.pdf" class="thumbnail"><img src="images/infuser.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://arxiv.org/pdf/2203.08715.pdf">Multiscale Sensor Fusion and Continuous Control with Neural CDEs</a></h3>
<p>
Sumeet Singh, Francis McCann Ramirez, Jacob Varley, Andy Zeng, Vikas Sindhwani<br>
<i>IEEE International Conference on Intelligent Robots and Systems (IROS) 2022</i><br>
<a href="https://arxiv.org/pdf/2203.08715.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/pdf/2203.01983.pdf" class="thumbnail"><img src="images/implicit-kinematic-policies.gif" alt="" height="135px"/></a>
<div class="project-description">
<h3><a href="https://arxiv.org/pdf/2203.01983.pdf">Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning</a></h3>
<p>
Aditya Ganapathi, Pete Florence, Jake Varley, Kaylee Burns, Ken Goldberg, Andy Zeng<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2022</i><br>
<a href="https://sites.google.com/view/implicit-kinematic-policies">Webpage</a> •
<a href="https://arxiv.org/pdf/2203.01983.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/pdf/2202.00868.pdf" class="thumbnail"><img src="images/virdo-sim.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://arxiv.org/pdf/2202.00868.pdf">VIRDO: Visio-Tactile Implicit Representations of Deformable Objects</a></h3>
<p>
Youngsun Wi, Pete Florence, Andy Zeng, Nima Fazeli<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2022</i><br>
<a href="https://arxiv.org/pdf/2202.00868.pdf">PDF</a> •
<a href="https://github.com/MMintLab/VIRDO">Code</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/pdf/2109.07578.pdf" class="thumbnail"><img src="images/multitask-transporter.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://arxiv.org/pdf/2109.07578.pdf">Multi-Task Learning with Sequence-Conditioned Transporter Networks</a></h3>
<p>
Michael H. Lim, Andy Zeng, Brian Ichter, Maryam Bandari, Erwin Coumans, Claire Tomlin, Stefan Schaal, Aleksandra Faust<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2022</i><br>
<a href="https://arxiv.org/pdf/2109.07578.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://arxiv.org/pdf/2110.04367.pdf" class="thumbnail"><img src="images/hybrid-random-features.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://arxiv.org/pdf/2110.04367.pdf">Hybrid Random Features</a></h3>
<p>
Krzysztof Choromanski, Haoxian Chen, Han Lin, Yuanzhe Ma, Arijit Sehanobish, Deepali Jain, Michael S Ryoo, Jake Varley, Andy Zeng, Valerii Likhosherstov, Dmitry Kalashnikov, Vikas Sindhwani, Adrian Weller<br>
<i>The International Conference on Learning Representations (ICLR) 2022</i><br>
<a href="https://arxiv.org/pdf/2110.04367.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://implicitbc.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/teaser-ibc.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://implicitbc.github.io/">Implicit Behavioral Cloning</a></h3>
<p>
Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, Jonathan Tompson<br>
<i>Conference on Robot Learning (CoRL) 2021</i><br>
<a href="https://implicitbc.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2109.00137.pdf">PDF</a> •
<a href="https://github.com/google-research/ibc">Code</a> •
<a href="https://ai.googleblog.com/2021/11/decisiveness-in-imitation-learning-for.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://x-irl.github.io/" class="thumbnail"><img src="images/xirl-overlay.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://x-irl.github.io/">XIRL: Cross-Embodiment Inverse Reinforcement Learning</a></h3>
<p>
Kevin Zakka, Andy Zeng, Pete Florence, Jonathan Tompson, Jeannette Bohg, Debidatta Dwibedi<br>
<i>Conference on Robot Learning (CoRL) 2021</i><br>
<font color="49bf9"><i>★ Best Paper Award Finalist, CoRL ★</i></font><br>
<a href="https://x-irl.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2106.03911.pdf">PDF</a> •
<a href="https://github.com/google-research/google-research/tree/master/xirl">Code</a> •
<a href="https://github.com/kevinzakka/x-magical">Benchmark</a> •
<a href="https://ai.googleblog.com/2022/02/robot-see-robot-do.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://spatial-intention-maps.cs.princeton.edu/" class="thumbnail"><img src="images/teaser-spatial-intention-maps.png" alt="" /></a>
<div class="project-description">
<h3><a href="https://spatial-intention-maps.cs.princeton.edu/">Spatial Intention Maps for Multi-Agent Mobile Manipulation</a></h3>
<p>
Jimmy Wu, Xingyuan Sun, Andy Zeng, Shuran Song, Szymon Rusinkiewicz, Thomas Funkhouser<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2021</i><br>
<a href="https://spatial-intention-maps.cs.princeton.edu/">Webpage</a> •
<a href="https://arxiv.org/pdf/2103.12710.pdf">PDF</a> •
<a href="https://github.com/jimmyyhwu/spatial-intention-maps">Code</a> •
<a href="https://engineering.princeton.edu/news/2022/01/25/picking-trash-robots-pick-new-approaches-work">Princeton News</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://berkeleyautomation.github.io/bags/" class="thumbnail"><img src="images/teaser-deformable-transporter.gif" alt="" /></a>
<div class="project-description">
<h3><a href="https://berkeleyautomation.github.io/bags/">Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks</a></h3>
<p>
Daniel Seita, Pete Florence, Jonathan Tompson, Erwin Coumans, Vikas Sindhwani, Ken Goldberg, Andy Zeng<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2021</i><br>
<a href="https://berkeleyautomation.github.io/bags/">Webpage</a> •
<a href="https://arxiv.org/pdf/2012.03385.pdf">PDF</a> •
<a href="https://github.com/DanielTakeshi/deformable-ravens">Code</a> •
<a href="https://ai.googleblog.com/2021/05/learning-to-manipulate-deformable.html">Google AI Blog</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://transporternets.github.io/" class="thumbnail">
<video playsinline="" muted="" autoplay="" loop="" width="180px">
<source src="images/transporter-nets-animation.mp4" type="video/mp4">
</video>
</a>
<div class="project-description">
<h3><a href="https://transporternets.github.io/">Transporter Networks: Rearranging the Visual World for Robotic Manipulation</a></h3>
<p>
Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, Johnny Lee<br>
<i>Conference on Robot Learning (CoRL) 2020</i><br>
<font color="49bf9"><i>★ Plenary Talk, Best Paper Presentation Award Finalist, CoRL ★</i></font><br>
<a href="https://transporternets.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/2010.14406.pdf">PDF</a> •
<a href="https://github.com/google-research/ravens">Code</a> •
<a href="https://ai.googleblog.com/2021/02/rearranging-visual-world.html">Google AI Blog</a> •
<a href="https://venturebeat.com/2020/10/28/googles-transporter-networks-learn-to-stack-blocks-and-assemble-mouthwash-kits-from-as-few-examples/">VentureBeat</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://spatial-action-maps.cs.princeton.edu/" class="thumbnail"><img src="images/teaser-spatial-action-maps.jpg" alt="" /></a>
<div class="project-description">
<h3><a href="https://spatial-action-maps.cs.princeton.edu/">Spatial Action Maps for Mobile Manipulation</a></h3>
<p>
Jimmy Wu, Xingyuan Sun, Andy Zeng, Shuran Song, Johnny Lee, Szymon Rusinkiewicz, Thomas Funkhouser<br>
<i>Robotics: Science and Systems (RSS) 2020</i><br>
<a href="https://spatial-action-maps.cs.princeton.edu/">Webpage</a> •
<a href="https://arxiv.org/pdf/2004.09141.pdf">PDF</a> •
<a href="https://github.com/jimmyyhwu/spatial-action-maps">Code</a> •
<a href="https://engineering.princeton.edu/news/2022/01/25/picking-trash-robots-pick-new-approaches-work">Princeton News</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://graspinwild.cs.columbia.edu/" class="thumbnail"><img src="images/teaser-grasp-in-wild.jpg" alt="" /></a>
<div class="project-description">
<h3><a href="https://graspinwild.cs.columbia.edu/">Grasping in the Wild: Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations</a></h3>
<p>
Shuran Song, Andy Zeng, Johnny Lee, Thomas Funkhouser<br>
<i>IEEE International Conference on Intelligent Robots and Systems (IROS) 2020</i><br>
<i>IEEE Robotics and Automation Letters (RA-L) 2020</i><br>
<a href="https://graspinwild.cs.columbia.edu/">Webpage</a> •
<a href="https://arxiv.org/pdf/1912.04344.pdf">PDF</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://form2fit.github.io" class="thumbnail"><img src="images/teaser-kitting.jpg" alt="" /></a>
<div class="project-description">
<h3><a href="https://form2fit.github.io/">Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly</a></h3>
<p>
Kevin Zakka, Andy Zeng, Johnny Lee, Shuran Song<br>
<i>IEEE International Conference on Robotics and Automation (ICRA) 2020</i><br>
<font color="49bf9"><i>★ Best Paper in Automation Award Finalist, ICRA ★</i></font><br>
<a href="https://form2fit.github.io/">Webpage</a> •
<a href="https://arxiv.org/pdf/1910.13675.pdf">PDF</a> •
<a href="https://github.com/kevinzakka/form2fit">Code</a> •
<a href="https://ai.googleblog.com/2019/10/learning-to-assemble-and-to-generalize.html">Google AI Blog</a> •
<a href="https://venturebeat.com/2019/10/31/watch-googles-ai-teach-a-picker-robot-to-assemble-objects/">VentureBeat</a> •
<a href="https://www.youtube.com/watch?v=O8l4Kn-j-5M">2 Minute Papers</a>
</p>
</div>
</div>
<div class="list-item publication" data-category="publication">
<a href="https://sites.google.com/view/cleargrasp" class="thumbnail"><img src="images/teaser-cleargrasp.gif" alt="" /></a>