A photo I took while at RSS 2022 in New York City, on the dinner cruise arranged by the conference.

From June 27 to July 01 this year, I attended the latest the Robotics: Science and Systems conference. This was the first in-person RSS after two virtual editions in 2020 and 2021. I don’t publish in RSS frequently. I only have one RSS paper, VisuoSpatial Foresight, from 2020. I was attending mainly because of its nearby location in New York City and because I was invited to attend RSS Pioneers, which is a unique event arranged by this conference.

I’m well aware that this report, like the one I wrote for ICRA 2022, is coming well after the conference has ended. Things have been extremely busy on my end but I will try and improve the turnaround time between attending and reporting about a conference.

RSS 2022: Pioneers

For the first day, I arrived on campus for RSS Pioneers, which happened the day before the “official” RSS conference. To quote from the website:

RSS Pioneers is an intensive workshop for senior Ph.D. students and early career researchers in the robotics community. Held in conjunction with the main Robotics: Science and Systems (RSS) conference, each year the RSS Pioneers brings together a cohort of the world’s top early-career researchers. The workshop aims to provide these promising researchers with networking opportunities and help to navigate their next career stages, and foster creativity and collaboration surrounding challenges in all areas of robotics. The workshop will include a mix of research and career talks from senior scholars in the field from both academia and industry, research presentations from attendees and networking activities.

I arrived early as usual to meet the sign language interpreters, which I had requested well in advance. (Columbia University paid the bill since they were hosting the conference.) We had some minor hiccups at the start when the security guards wouldn’t let the interpreters inside with me. Columbia Mechanical Engineering professor Matei Ciocarlie was the local arrangements chair of RSS and thus in charge of much of the day-to-day affairs, and he kept trying to explain to the security guards why there were there and why we all had to be allowed inside. Eventually he convinced the security guards, and I arrived just in the nick of time.

During RSS Pioneers, the 30 or so “Pioneers” gave brief presentations of our research. I presented my recent theme of robot learning for deformable manipulation. There were also talks from professors, which for Pioneers was an interesting mix of both technical talks and high-level thoughts. Some of the “high-level” aspects of the talks included de-emphasizing rankings of the school, the different research cultures of the United States and Europe, how (not) to compare one’s research with others, and dealing with publish-or-perish. Throughout the day, we had brief social breaks, and ultimately, the real value of this event is probably that we get to know other Pioneers and increase our connections with the robotics field. Personally, it took me a while to view myself as a roboticist. Overall I’m happy I attended this year’s edition.

RSS 2022: The Conference

The actual conference and “conference workshops” (hereafter, just “workshops”) started the next day, with the first and fifth days for workshops. An interesting workshop I attended was about “Overlooked Aspects on Imitation Learning.” In general, as per my rule for attending these events, I tried not to get bogged down by attending every talk in a workshop, and instead I spent some time in a common room area where they had food and snacks, to see if other attendees were available for impromptu chats.

In between the two workshop days, the conference had the usual set of keynotes and paper presentations. A lot of the interesting papers to me were ones that I already knew of in advance (thanks to arXiv, Twitter, and the like) such as DextAIRity, Iterative Residual Policy, and Autonomously Untangling Long Cables, all three of which garnered awards (or were finalists for them). An interesting paper which I did not know about in advance was You Only Demonstrate Once. From a technical perspective, I think it’s a very nice systems paper which combines a variety of techniques: using specialized objects coordinate spaces (called “NUNOCS” in the paper), a motion tracker, category-level behavioral cloning, etc. I should also give a shout-out to the FlowBot3D paper from our lab, which was one of the best paper finalists. I am not an author, but my ToolFlowNet paper for the upcoming Conference on Robot Learning (CoRL) builds upon FlowBot3D with similar ideas.

A unfortunate thing is that, as with ICRA earlier this year, we continually ran into technical problems with speakers unable to reliably connect their laptops to the projectors. In one particularly unfortunate oral presentation, the projector strangely turned off, then on, at regular intervals. I counted an interval time of about 10 seconds where, without fail, the projector would turn off and on, almost perfectly on cue! I felt very sorry for the speaker. Is there any way that we can improve the state of affairs for connecting to projectors? It seems to be a universal problem.

In addition to talks, poster sessions, and coffee break chats, since RSS was at Columbia University. I was fortunate to get a semi-private tour of the Columbia robotics lab. I saw FlingBot and DextAIRity in action. I thank the members of Shuran Song’s lab who gave me this tour. Having a set of coordinating UR5 arms in one setup has been great for their research acceleration.

RSS does not provide lunch or dinner for each day, but as is typical in academic conferences in my circle, one of the nights is dedicated to a “fancy” dinner event. For this year’s edition, RSS had dinner on a cruise ship which took us around lower Manhattan. The food was incredibly delicious and diverse, and perhaps the best I have eaten at a conference – there were salads, vegetarian dishes, fish, chicken, fancy desserts, and a bunch of stuff I can’t recall. Obviously this service doesn’t come cheap. I heard it’s about 150 USD per person. Aided by the interpreter who did a tremendous job in the noisy context, I had a bunch of intellectually stimulating conversations which I recall included robotics (obviously), politics (should the US military fund robotics research), and my future job plans (one can only be a postdoc for so long).

The conference also had a town hall, to solicit feedback from us. This is a good thing, and I appreciate RSS for doing this. Some of the questions had to do with the nature of what a “systems” paper means. Other questions had to do with reducing stress on students and researchers. One of the paper awards for RSS is a “Best systems paper award in honor of Seth Teller.” Seth Teller was a professor at MIT and passed away in 2014 from suicide.

From the closing ceremonies, we learned that RSS 2023 will be in Daegu, South Korea next year, which is the first time it’s being held in Asia. This is a good thing for the sake of balance, given the recent US-heavy tilt on conferences. It will pose some issues for me if I were to request sign language interpreters, but I am OK with balancing the location of conferences since we need to consider the full community’s needs (not just mine) when picking locations.

RSS 2022: Overall Impressions

I was fortunate to attend RSS in person for the first time in my career. The conference is much smaller compared to ICRA and IROS and is single-track, which meant I did not feel overwhelmed with competing priorities. I exceeded my expectations in meeting people and understanding the work they do. Certainly, being part of RSS Pioneers helped, and I would recommend this to senior PhD students and postdocs.

Columbia University’s in a fantastic location in New York City which makes travel to and from it easier. The subway system made transportation within the city easier, and there’s a lot of food nearby with a range of prices, not to mention attractions and museums. (Alas, I wish I had the time to spend a few extra days in NYC to explore it more…) The location also meant that, as is typically the case in big American cities, securing sign language interpreters generally isn’t too difficult. Of course, there are some inevitable challenges with space that come with NYC, and we might have seen that with the poster sessions at RSS, since they were in narrow ramps and stairs in one building. In contrast, ICRA 2022 in Philadelphia was not at the University of Pennsylvania campus but at the Pennsylvania Convention Center with massive space.

RSS has some differences in terms of paper structure and reviewing as compared to ICRA and IROS. At RSS, there’s a longer page limit requirement, and since page limits in practice serve as lower bounds (since no one wants to be viewed as lacking content) that means papers have more material. At RSS there’s a slightly stricter acceptance rate and a more involved reviewing process (often with rebuttals) compared to other conferences. However, I’m well aware that this is subject to enormous variance in paper quality. But my point is: RSS papers tend to be high quality so if I see that a paper was presented there, it already hits some minimum quality threshold for me.

In terms of the content, RSS is becoming increasingly similar to the [Conference on Robot Learning (CoRL)]. A lot of RSS papers use machine learning. Matei Ciocarlie, the local arrangements chair, seems to have a similar interpretation and said to me: “oh, they [the RSS workshops] were all in robot learning …” A lot of RSS and CoRL papers could be interchangeable, in that papers accepted to RSS, assuming they have some robot learning aspect, could be accepted at CoRL and possibly vice versa. I have frequently seen cases when researchers submit papers to RSS, get rejected, improve the paper (to varying extents…), and get the paper accepted to CoRL. The reverse isn’t generally true since CoRL rejections tend to go to the following ICRA due to the timing.

Given its full name as Robotics: Science and Systems, this might mean RSS papers have more of a physical robotics “system” aspect as compared to CoRL, though I am not sure if this applies in practice. UIUC Professor Kris Hauser chaired this edition’s conference and he wrote this Medium article on what it means to write a systems-related robotics paper. Whether the community follows through on this is another matter, and there were some questions about the “systems” aspect research in the RSS town hall.

In sum, to those who helped organize this year’s RSS conference, thank you very much!! I feel very fortunate to have attended and I hope to do so again in future editions.

Finally, regarding CoRL, that conference is happening in a few days in Auckland, New Zealand. If you are attending, I will also be there and am happy to chat …