Trick or treat, Atlas!

Sparsh 개요 Meta의 Sparsh는 비전 기반 촉각 센서를 위한 자기 지도 학습(Self-supervised Learning, SSL) 모델을 제안하는 연구입니다. 촉각센서 정보를 이미지화 -> 이미지 자기지도학습 방식을 촉각에 적용 -> 촉각 자기지도학습 생성형 AI를 촉각 영역까지 확장하여 로봇의 인지 및 조작 능력을 대폭 향상 했다는 점에서 의의를 지닙니다. LLM: 언어 자기지도학습 JEPA: 영상 자기지도학습 Sparsh: 촉각 자기지도학습 촉각 센서(DIGIT, GelSight 등)는 로봇이 물체를 다룰 때 시각 정보만으로는 얻기 어려운 촉각 피드백을 제공하여 로봇 조작 능력을 크게 향상시킵니다. 그러나 기존의 접근 방식은 특정 작업과 센서에 맞춘 모델을 사용하며, 이러한 모델을 학습시키기 위해서는 많은 양의 라벨링된 데이터가 필요합니다. 이는 데이터 수집...

[요약] Waymo에서 멀티모달 LLM Gemini nano를 이용하여 자율주행을 구현하였고, 기존 Waymo 자율주행 모델인 Waymoformer 대비 우수한 성능을 보임 LLM의 reasoning 성능향상을 위해 사용되는 학습/프로프트 테크닉인 Chain of Thought 를 이용하여 차량 Path Planning을 구현함. Transformer 기반의 LLM의 언어 성능에 적용되던 Scaling law가 자율주행에도 적용 됨을 확인, 규모 경쟁이 발생할 가능성이 높음 End-to-End Multimodal Models: The Future of Autonomous Driving? The rapid evolution of AI has been nothing short of transformative, and its impact on industries like autonomous driving is becoming increasingly evident. A striking example is the shift towards End-to-End multimodal models for autonomous driving, a trand that has gained momentum thanks to companies like Tesla and Waymo. The ChatGPT Revolution and its Influence To understand how we arrived at this point, we need to revisit the rise of transformer-based models like OpenAI's ChatGPT. ChatGPT demonstrated the incredible potential of large language models (LLMs) to to process and generate human-like text by learning from vast amount of data. The success sparked a broader interest in applying similar architectures beyond natural language processing (NLP), including in fields like computer vision and robotics. Tesla took note of this paradigm shift. inspired by the success of transformer-based models in NLP, Tesla restructured its Full Self-Driving (FSD) system into an End-to-End model. However, instead of processing text, Tesla's model processes images from the cameras mounted on the vehicle. The goal? To generate driving path as output, replacing traditional rule-based systems with a more holistic approach that directly maps raw sensor data to driving actions. The Rise of End-to-End Multimodal Models While the concept of E2E models was gaining traction by late 2023, there was still limited clarity on the exact architectures being used by industry leaders. However, just before the end of 2024, Waymo made a significant contribution to this space by publishing a detailed paper on their End-to-End Multimodal Model for Autonomous Driving, known as EMMA EMMA represents a major leap forward in autonomous driving technology. Built on a ...