{"id":6041,"date":"2024-11-24T11:43:17","date_gmt":"2024-11-24T03:43:17","guid":{"rendered":"https:\/\/autowise.ai\/?p=6041"},"modified":"2024-11-24T11:43:17","modified_gmt":"2024-11-24T03:43:17","slug":"tech-unpacked%ef%bd%9cai-safety-officer-empowers-autonomous-driving-with-multimodal-large-model-technology-2-2","status":"publish","type":"post","link":"https:\/\/autowise.ai\/en\/2024\/11\/24\/tech-unpacked%ef%bd%9cai-safety-officer-empowers-autonomous-driving-with-multimodal-large-model-technology-2-2\/","title":{"rendered":"Tech Unpacked\uff5cAI Safety Officer Empowers Autonomous Driving with Multimodal Large Model Technology"},"content":{"rendered":"[vc_row type=”full_width_content” full_screen_row_position=”middle” column_margin=”default” column_direction=”default” column_direction_tablet=”default” column_direction_phone=”default” scene_position=”center” top_padding=”80px” left_padding_desktop=”15%” constrain_group_2=”yes” right_padding_desktop=”15%” top_padding_tablet=”60px” left_padding_tablet=”5%” constrain_group_4=”yes” right_padding_tablet=”5%” top_padding_phone=”30px” left_padding_phone=”20px” constrain_group_6=”yes” right_padding_phone=”20px” text_color=”dark” text_align=”left” row_border_radius=”none” row_border_radius_applies=”bg” overflow=”visible” advanced_gradient_angle=”0″ overlay_strength=”0.3″ gradient_direction=”left_to_right” shape_divider_position=”bottom” bg_image_animation=”none” gradient_type=”default” shape_type=””][vc_column column_padding=”no-extra-padding” column_padding_tablet=”inherit” column_padding_phone=”inherit” column_padding_position=”all” left_margin=”auto” constrain_group_2=”yes” right_margin=”auto” column_element_spacing=”default” background_color_opacity=”1″ background_hover_color_opacity=”1″ column_shadow=”none” column_border_radius=”none” column_link_target=”_self” max_width_desktop=”1000px” column_position=”default” advanced_gradient_angle=”0″ gradient_direction=”left_to_right” overlay_strength=”0.3″ width=”1\/1″ tablet_width_inherit=”default” tablet_text_alignment=”default” phone_text_alignment=”default” animation_type=”default” bg_image_animation=”none” border_type=”simple” column_border_width=”none” column_border_style=”solid” gradient_type=”default”][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
As a global leader in the commercialization of autonomous driving, Autowise.ai has deployed multiple autonomous driving products across over 30 cities worldwide. To meet the varied demands of multi-region, multi-scenario, and multi-product operations, Autowise has integrated cutting-edge AI technologies and algorithms. Their new generation 1vN autonomous driving remote operations system incorporates advanced multimodal large-model technology, acting as an \u201cAI Safety Officer\u201d to bolster safety for autonomous vehicles with robust scene understanding and accurate decision-making capabilities.<\/div>\n<\/div>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
AI Safety Officer Uses Multimodal Large Model to Analyze Real-Time Image Data and Identify Potential Hazards<\/strong><\/div>\n<\/div>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
The multimodal large model represents a significant advancement in AI, integrating various data forms\u2014text, images, audio, and video\u2014into a unified framework that enables cross-modal understanding and generation. This technology supports diverse tasks, such as image description, visual Q&A, and video generation. In autonomous driving, the AI Safety Officer processes and analyzes camera images in real-time, identifying hazardous scenes and abnormal situations that the vehicle may encounter, thus ensuring safer operations.<\/div>\n<\/div>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
Tackling Long-Tail Scenarios Beyond Algorithmic Coverage<\/strong><\/div>\n<\/div>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
In addition to interacting with conventional road participants, autonomous vehicles must handle complex, rare “long-tail” scenarios\u2014situations that occur infrequently but pose significant challenges to autonomous systems. Autowise\u2019s autonomous sweepers, for example, often operate close to the curb, where they encounter unique challenges such as irregular potholes, temporary construction zones, or unexpected traffic incidents.<\/div>\n<\/div>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
These long-tail scenarios are critical because they represent rare yet potentially dangerous situations that an autonomous driving system must navigate. Human drivers rely on intuition and experience to handle these edge cases. However, autonomous systems need vast data and complex algorithms to simulate this decision-making process. Given the complexity of real-world conditions, algorithms alone cannot cover all scenarios. Here, the AI Safety Officer steps in, leveraging near-human knowledge and experience to ensure safe navigation.<\/div>\n<\/div>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
Real-World Applications of the AI Safety Officer: Scene Interpretation and Advisory Responses<\/strong><\/div>\n<\/div>\n[\/vc_column_text][nectar_video_player_self_hosted video_webm=”https:\/\/static-cdn.autowise.tech\/wp-content\/uploads\/2024\/11\/\u591a\u6a21\u6001125M.mp4″ player_functionality=”default” border_radius=”none” box_shadow=”none” advanced_gradient_angle=”0″ advanced_gradient_display_type=”linear”][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
Autowise has developed an abnormal dataset for decision-making and planning based on historical takeover data and closed-loop simulation data, leveraging model annotation and human alignment. Enriched with real-world operational data, the AI Safety Officer\u2019s multimodal large model will offer increasingly accurate scene interpretation and driving recommendations.<\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_column_text]Below are some examples illustrating its capabilities:[\/vc_column_text][vc_column_text]Sudden Traffic Accidents<\/em>[\/vc_column_text][image_with_animation image_url=”5822″ image_size=”full” animation_type=”entrance” animation=”Fade In” hover_animation=”none” alignment=”” border_radius=”none” box_shadow=”none” image_loading=”default” max_width=”100%” max_width_mobile=”default”][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
While an autonomous sweeper operates on a side lane, it encounters a sudden traffic accident\u2014say for example a delivery rider skids on a wet road. The system flags a “Traffic Accident” status code and alerts for possible remote intervention. Fortunately, the rider is unharmed, and the system advises slowing down and proceeding cautiously.<\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_column_text]Navigating Temporary Road Closuresand Construction Zones[\/vc_column_text][image_with_animation image_url=”5828″ image_size=”full” animation_type=”entrance” animation=”Fade In” hover_animation=”none” alignment=”” border_radius=”none” box_shadow=”none” image_loading=”default” max_width=”100%” max_width_mobile=”default”][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
The sweeper encounters both construction work on the side lane and overhead work on the main road, with traffic police directing vehicles. The system flags “Traffic Police Control” and “Construction Area” status codes and prompts for potential remote intervention. With remote assistance, the sweeper safely reroutes through an adjacent lane.<\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
Emergency Vehicle Interaction\u000b<\/em><\/strong><\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][image_with_animation image_url=”5832″ image_size=”full” animation_type=”entrance” animation=”Fade In” hover_animation=”none” alignment=”” border_radius=”none” box_shadow=”none” image_loading=”default” max_width=”100%” max_width_mobile=”default”][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
While performing curbside sweeping, the vehicle spots an ambulance with flashing lights parked roadside, indicating a nearby emergency. The system raises an “Emergency Vehicle” status code, prompting remote intervention and, when safe, advises the sweeper to slow down and yield.<\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
Interacting with Vulnerable Road Users<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][image_with_animation image_url=”5835″ image_size=”full” animation_type=”entrance” animation=”Fade In” hover_animation=”none” alignment=”” border_radius=”none” box_shadow=”none” image_loading=”default” max_width=”100%” max_width_mobile=”default”][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
In a narrow lane, due to a parked vehicle, the sweeper detects children and crouched pedestrians nearby, raising a “High-Risk Pedestrians” status code. It advises reduced speed and alerts for possible remote intervention. Using high-precision perception, the sweeper makes a slight avoidance maneuver and proceeds slowly.<\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
AI Provides Safety Backbone, While Human Operators Offer Additional Support for Emergency Situations<\/strong><\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
In specific emergency or extreme scenarios, the AI Safety Officer prompts human remote safety operators for intervention. These operators receive real-time video feeds (front and 360-degree views) and sensor data (high-precision maps, obstacle detection, path planning) to assess the vehicle\u2019s environment and provide support to help the vehicle safely navigate. Autowise\u2019s autonomous vehicles currently support various remote-assist driving options\u2014including reference-line, steering, app-based, and cockpit controls\u2014providing an extra layer of safety in emergencies.<\/div>\n<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_gallery type=”nectarslider_style” images=”5838,5841″ image_loading=”default” bullet_navigation_style=”see_through” onclick=”link_no”][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
The AI Safety Officer, powered by the cloud-based large model, significantly enhances the autonomous vehicle\u2019s ability to recognize and understand long-tail scenarios. This improvement also increases the efficiency of human remote operators, laying a strong technical foundation for the ongoing expansion of 1vN-scale remote operations and broadening Autowise\u2019s global commercial reach.<\/div>\n<\/div>\n<\/section>\n
<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][vc_column_text css_animation=”fadeInUp” max_width=”1000px”]\n
\n
\n
\n
\n
\n
\n
Looking ahead, Autowise will continue to adopt and integrate leading-edge AI technologies and algorithms, innovating to improve the safety and reliability of autonomous driving technology. These advancements will form a solid foundation for the rapid commercialization of autonomous driving, contributing to the global transformation of autonomous driving and AI industries.<\/div>\n<\/div>\n<\/section>\n
<\/div>\n<\/section>\n<\/section>\n<\/section>\n<\/section>\n
\n
<\/section>\n<\/section>\n[\/vc_column_text][\/vc_column][\/vc_row][vc_row type=”in_container” full_screen_row_position=”middle” column_margin=”default” column_direction=”default” column_direction_tablet=”default” column_direction_phone=”default” scene_position=”center” text_color=”dark” text_align=”left” row_border_radius=”none” row_border_radius_applies=”bg” overflow=”visible” overlay_strength=”0.3″ gradient_direction=”left_to_right” shape_divider_position=”bottom” bg_image_animation=”none”][vc_column column_padding=”no-extra-padding” column_padding_tablet=”inherit” column_padding_phone=”inherit” column_padding_position=”all” column_element_spacing=”default” background_color_opacity=”1″ background_hover_color_opacity=”1″ column_shadow=”none” column_border_radius=”none” column_link_target=”_self” column_position=”default” gradient_direction=”left_to_right” overlay_strength=”0.3″ width=”1\/1″ tablet_width_inherit=”default” tablet_text_alignment=”default” phone_text_alignment=”default” animation_type=”default” bg_image_animation=”none” border_type=”simple” column_border_width=”none” column_border_style=”solid”][\/vc_column][\/vc_row][vc_row type=”in_container” full_screen_row_position=”middle” column_margin=”default” column_direction=”default” column_direction_tablet=”default” column_direction_phone=”default” scene_position=”center” text_color=”dark” text_align=”left” row_border_radius=”none” row_border_radius_applies=”bg” overflow=”visible” overlay_strength=”0.3″ gradient_direction=”left_to_right” shape_divider_position=”bottom” bg_image_animation=”none”][vc_column column_padding=”no-extra-padding” column_padding_tablet=”inherit” column_padding_phone=”inherit” column_padding_position=”all” column_element_spacing=”default” background_color_opacity=”1″ background_hover_color_opacity=”1″ column_shadow=”none” column_border_radius=”none” column_link_target=”_self” column_position=”default” gradient_direction=”left_to_right” overlay_strength=”0.3″ width=”1\/1″ tablet_width_inherit=”default” tablet_text_alignment=”default” phone_text_alignment=”default” animation_type=”default” bg_image_animation=”none” border_type=”simple” column_border_width=”none” column_border_style=”solid”][image_with_animation image_size=”full” animation_type=”entrance” animation=”Fade In” hover_animation=”none” alignment=”” border_radius=”none” box_shadow=”none” image_loading=”default” max_width=”100%” max_width_mobile=”default”][\/vc_column][\/vc_row]\n","protected":false},"excerpt":{"rendered":"

[vc_row type=”full_width_content” full_screen_row_position=”middle” column_margin=”default” column_direction=”default” column_direction_tablet=”default” column_direction_phone=”default” scene_position=”center” top_padding=”80px” left_padding_desktop=”15%” constrain_group_2=”yes” right_padding_desktop=”15%” top_padding_tablet=”60px” left_padding_tablet=”5%” constrain_group_4=”yes” right_padding_tablet=”5%” top_padding_phone=”30px” left_padding_phone=”20px” constrain_group_6=”yes” right_padding_phone=”20px” text_color=”dark” text_align=”left” row_border_radius=”none” row_border_radius_applies=”bg” overflow=”visible” advanced_gradient_angle=”0″ overlay_strength=”0.3″ gradient_direction=”left_to_right” shape_divider_position=”bottom” bg_image_animation=”none”…<\/p>\n","protected":false},"author":2,"featured_media":5838,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[],"class_list":{"0":"post-6041","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-news"},"_links":{"self":[{"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/posts\/6041","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/comments?post=6041"}],"version-history":[{"count":2,"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/posts\/6041\/revisions"}],"predecessor-version":[{"id":6324,"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/posts\/6041\/revisions\/6324"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/media\/5838"}],"wp:attachment":[{"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/media?parent=6041"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/categories?post=6041"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/autowise.ai\/en\/wp-json\/wp\/v2\/tags?post=6041"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}