Cyberphysical Smart Cities Infrastructures

House3D [69], AI2‐THOR [70], Gibson [71], CHALET [72], MINOS [73], and Habitat [74] are some of the popular simulators for the embodied AI studies. These platforms vary with respect to the 3D environments they use, the tasks they can handle, and the evaluation protocols they provide. These simulators support different sensors such as vision, depth, touch, and semantic segmentation.

In this chapter, we mainly focus on MINOS and Habitat since they provide more customization abilities (number of sensors, their positions, and their parameters) and are implemented in a loosely coupled manner to generalize well to new multisensory tasks and environments. As their API can be used to define any high‐level task and the material, object clutter variation and many more can be programmatically configured for the environment. They both support navigation with both continuous and discrete state spaces. Also, for the purpose of their benchmarks, all the actuators are noiseless, but they both have the ability to enable noises if desired [75].

In the last section, we saw numerous task definitions and how they each can be tackled by the agents. So, before jumping into MINOS and Habitat simulators and reviewing them, let us first get more familiarized with the three main goal‐directed navigation tasks, namely, PointGoal Navigation, ObjectGoal Navigation, and RoomGoal Navigation.

In PointGoal Navigation, an agent is appeared at a random starting position and orientation in a 3D environment and is asked to navigate to target coordinates that are given relative to the agent's position. The agent can access its position via an indoor GPS. There exists no ground‐truth map, and the agent must only use its sensors to do the task. The scenarios start the same for ObjectGoal Navigation and RoomGoal Navigation as well; however, instead of coordinates, the agent is asked to find an object or go to a specific room.

MINOS simulator provides access to 45 000 three‐dimensional models of furnished houses with more than 750 K rooms of different types available in the SUNCG [76] dataset and 90 multi‐floor residences with approximately 2000 annotated room regions that are in the Matterport3D [77] dataset by default. Environments in Matterport3D are more realistic looking than the ones in SUNCG. MINOS simulator can approximately reach hundreds of frames per second on a normal workstation.

To benchmark the system, the authors studied four navigation algorithms, three of which were based on asynchronous advantage actor‐critic (A3C) approach [78] and the remaining one was direct future prediction (DFP) [79].

The most basic one among the algorithms was feedforward A3C. In this algorithm, a feedforward convolutional neural network (CNN) model is employed as the function approximator to learn the policy along with the total value function that is the expected sum of rewards from the current timestamp until the end of the episode. The second one was LSTM A3C that used an LSTM model with the feedforward A3C acting as a simple memory. Next was UNREAL, an LSTM A3C model boosted with auxiliary tasks such as value function replay and reward prediction. Last but not the least, the DFP algorithm was employed that can be considered as Monte Carlo RL [80] with a decomposed reward.

The authors benchmarked these algorithms on PointGoal and RoomGoal tasks and found out that, firstly, the naive feedforward algorithm fails to learn any useful representation and, secondly, in small environments, DFP performs better while in big and more complex environments, UNREAL beat the others.

Habitat was designed and built in a way to provide the maximum customizability in terms of the datasets that can be used and how the agents and the environment can be configured. That being said, Habitat works with all the major 3D environment datasets without a problem. Moreover, it is extremely fast in comparison with other simulators. AI2‐THOR and CHALET can get to an fps of roughly 10, MINOS and Gibson can get to around a hundred, and House3D yields 300 fps in the best case, while Habitat is capable of getting up to 10 000 fps. It also provides a more realistic collision model in which if a collision happens, the agent can be moved partially or not at all in the intended direction.

To benchmark Habitat, the owners employed a few naive algorithm baselines, proximal policy optimization (PPO) [81] as the representer of learning algorithms versus ORB‐SLAM2 [82, 83] as the chosen candidate for non‐learning agents, and tested them on the PointGoal Navigation task on Gibson and Matterport3D. They used Success weighted by Path Length (SPL) [84] as the metric for their performance. The PPO agent was tested with different levels of sensors (e.g. no visual sensor, only depth, only RGB, and RGBD) to perform an ablation study and find the proportion in which each sensor helps the progress. SLAM agents were given RGBD sensors in all the episodes.

The authors found out that first, PPO agents with only RGB perform as bad as agents with no visual sensors. Second, all agents perform better and generalize more on Gibson rather than Matterport3D since the size of environments in the latter is bigger. Third, agents with only depth sensors generalize across datasets the best and can achieve the highest SPL. However, most importantly, they realized that unlike what has been mentioned in the previous work, if the PPO agent learns long enough, it will eventually outperform the traditional SLAM pipeline. This finding was only possible because the Habitat simulator was fast enough to train PPO agents for 75 million time steps as opposed to only 5 million time steps in the previous investigations.

Consciousness has always been considered as the ultimate characteristic for true intelligence. Qualia [85, 86] is the philosophical view of consciousness and is related to the subjective sensory qualities like “the redness of red” that humans have in their mind. If at some point machines can understand this concept and objectively measure such things, then the ultimate goal can be marked as accomplished.

Robots still struggle at performing a wide spectrum of tasks effortlessly and smoothly, and this is mainly due to actuator technology as currently most electrical motors are used. Advances in artificial muscles and skin sensors that could cover the entire embodiment of the agent would be essential to fully mitigate the human experience in the real world and eventually unlock the desired cognition [87].

One more key component for cognition is the ability to grow and evolve over time 88, 90. It is easy to evolve the agent's controller via an evolutionary algorithm, but it is not enough. If we aim to have completely different agents, we might as well give them the ability to evolve in terms of embodiment and the sensors as well. This again requires the abovementioned artificial cell organism to encode different physical attributes in them and flip them slightly over time. Of course, we are far from this to become reality, but it is always good to know the furthermost step that has to be done one day.

Embodied AI is the field of study that takes us one step closer to the true intelligence. It is a shift from Internet AI toward embodiment intelligence that tries to exploit the multisensory abilities of agents such as vision, hearing, and touch, together with language understanding and reinforcement learning attempts to interact in the real world in a more sensible way. In this chapter, we tried to do a concise review of this field and its current advancements, subfields, and tools expecting that this would help accelerate future researches in this area.