Why the U.S. military is gung-ho about autonomous boats on RTInsights

Why the U.S. military is gung-ho about autonomous boats on RTInsights

I am continuing to publish my articles on RTInsights; here is a taste of my article on autonomous naval vessels. To read the full article go to https://www.rtinsights.com/why-the-u-s-military-is-gung-ho-about-autonomous-boats/

The U.S. Navy is investing in autonomous boats for reconnaissance, payload delivery, and acting as decoys. but it is clear they can do more than that both militarily and eventually in the private sector.

Unlike autonomous motorcycles, autonomous boats make sense at first glance, as well as after considering them for several hours. Boats are useful for transporting goods and people over long distances. Given the terrors of the sea, the promise of a boat in which fewer humans are at risk is a real boon.

Given the breadth of the subject of autonomous boats, it is best to discuss it in parts. To begin, I’ll look at the United States Armed Forces’ work in naval automation, especially because the technology they develop can be applied to civilian use.

Why autonomous motorcycles are destined for the road

Why autonomous motorcycles are destined for the road

RTInsights just published my article on automating the motorcycle. To read the full article go to https://www.rtinsights.com/why-autonomous-motorcycles-are-destined-for-the-road/.


While autonomous motorcycles aren’t keeping pace with self-driving automobiles, automating parts of the motorcycle will make them safer and has applications in other industries.

It seems ridiculous to even suggest a self-driving motorcycle. If you are in an autonomous car then you are free to watch videos or write reports. Trying the same on a motorcycle is difficult at best, dangerous at worst.

Although autonomous motorcycle technology may not allow motorcyclists to juggle on the go, they can make them safer and may have applications outside the vehicle market.

If you have ever lost your luggage

If you have ever lost your luggage

RTInsights is now publishing my articles! If you would like to see this week’s edition on how the IoT can help airline passengers track their luggage, check out my article at https://www.rtinsights.com/how-iot-technology-can-solve-the-lost-luggage-problem/

Here is just a taste of the article:

“Recently, Delta Airlines was scheduled to take my family from Boston to Detroit to Rome, so we could drive to Tuscany, but the trip went off the rails. To make a long story short, due to weather conditions we were rerouted onto several flights that did not go as planned. We finally had to run to another terminal to fly with Air France.

Air France took us to Rome — but without our bags. The people at baggage claim were pleasant but could not give us any information, so they tried to placate us by saying the bags would likely arrive on the first flight the following day. We took them at their word and cut our trip to Tuscany a day short so we could go with our bags.

The bags did not arrive and accurate information was not forthcoming. We went to Tuscany and spent the next four days calling Delta, Air France’s bag tracking office, and Alitalia’s Rome office and received different answers until our bags finally arrived. What happened?”

While you are waiting: Distributed Computing Networks

While you are waiting: Distributed Computing Networks

How do you turn this off?

Our vehicles are parked 95% of the time. Our laptops are not always in our laps. Even smartphones, which are now seen by many as vital, do not always spend time in our hands. Furthermore, more of our devices now go to sleep instead of turning off, and some like the Apple TV no longer turn off at all. What does this mean, and does it matter? Some programs such as CrashPlan take advantage of the idle time by performing tasks such as backups and virus scans. A more imaginative and useful route would be to donate a device’s unused processing power to take part in distributed computing.

Distributed computing is a process where people yield access to the computer processing power found within the computer processing unit (CPU) and/or graphics processing unit (GPU) within their devices to a third actor, who uses these distributed computers to complete larger tasks. This practice is already being used in a number of computers and supercomputers around the world. Right now, many scientists use “volunteer computing ” for tasks such as modeling protein folding, modeling the transmission of malaria, and enabling earthquake monitoring. Today many research projects get the processing power for free, but in the future people may expect payment in return. Luis Sarmenta of the MIT Computer Science and Artificial Intelligence Laboratory discusses what he calls “paid volunteer computing” and points out several issues to consider. Companies need to make sure there are no cheaters who falsely claim they are contributing processing power. Companies must also be vigilant against espionage, since someone who enters the network may try to gather information and sell it to a corporate competitor, leaving the network’s owner vulnerable to theft. Companies may have to encrypt their networks. There is also a risk to the contributor to the network. One’s device could be unknowingly added to an illicit network, such as a botnet that is used to send distributed denial-of-service (DDOS) attacks or spam e-mails. Finally, it is complicated how people should be paid considering the different kind of devices, how often they could contribute processing power, etc.
One example of a company that sells other users’ processing power is Peer Zone, which uses idle resources to test the performance of popular websites. They pay the people with the processing power on a per minute basis, regardless of the processing power of one’s computer, and limit the payment to $45/month. Paid volunteer computing has its growing pains.  CPUsage states that it has a glut of people willing to sell their CPUs’ idle time and a shortage of companies interested in it. Some, like Slicify, closed down after attempting to use other peoples’ idle time to compete with cloud computing companies such as Amazon and Microsoft. Perhaps the growing pains are due to the fact that this is a new market. Ultimately, paid volunteer computing could compete with immobile, expensive data centers because the companies do not own the computers and do not need the capital to buy the processing power. Paid volunteer computing may have the advantage in being more flexible then a data center.

The PlayStation 3’s Life with PlayStation client displaying a 3-D animation of a folding protein. The videogame console was once recruited by researchers to help model folding proteins for Stanford.

There is no reason why paid computing should be limited to computers. Stanford University’s Folding@home used the processing capacity of networked computers and PlayStation 3s to model protein folding to help researchers study diseases until 2012. While refrigerators are not known for having a number of processors, in 2014 a smart fridge sent spam as part of a botnet, suggesting that it has processing power to spare. As discussed in a previous blog, vehicles can also be added to a distributed computing network, since they are parked on average 95% of the time. Those who want to volunteer processing power must take care to leave electronic control units (ECUs) available for critical situations. While adding refrigerators and vehicles to distributed networks may seem unorthodox, they have plenty of computing power to spare!

Revenue from Data: Capitalism and Privacy and the Autonomous Automotive

This is a picture of the Waze app, a GPS navigation software used by many to navigate. By using the app users provide the app with more data

The rise of the smartphone created many new markets in terms of data gathering and analysis and provide novel uses for telephones. For example, the Global Positioning System (GPS) that comes pre-installed on many phones helps researchers track the mobility of people with Alzheimer’s Disease. MyFitnessPal allows you to track what you eat and how much you exercise on most mainstream phones, which could provide nutritionists and food companies with important data, if the user shares the data. Connected vehicle owners have the opportunity to make money by selling access to their data and their vehicles’ processing power.

The art of data mining is already well-known. Companies such as Facebook and Google collect vast amounts of data from various sources, analyze it, and sell access to the data. There is a saying in the technology industry: “If you are not paying for the product, you are the product.” Currently people usually mine vehicle data for purposes such as traffic analysis and developing self-driving technologies such as cruise control; later on marketers could use data from the vehicles to track how people decide to drive, seeing if they slow down around billboards, learn more about how they pick gas stations, etc. There is already a worldwide trend for car insurers to use car location data to set pricing. Right now programs such as Progressive Insurance’s Snapshot program are optional, but may become mandatory.

People could make a mint by selling access to the processing power in their vehicles’ computers. Vehicles have“electronic control units” (ECUs) inside them to handle functions such as transmission shifting and skid-control. The average car now has somewhere between 25 and 50 central processing units (CPUs) and the top of the line have more than 100. All this processing power is currently being used to run cars, but in the future it is possible that vehicles will have so much processing power that they will be able to “rent out” their unused data to other services, such as hospitals and animation studios. Connected Vehicles can also act as mobile data centers and thus capable of processing information remotely. In dangerous situations vehicular data centers could be more useful than stationary data centers, since they can be moved quickly.

While data mining and renting out access to a vehicle’s processing power can benefit owners and society alike, it raises issues regarding privacy and the resale of purchased data. People are already asking questions about how much data Facebook and other companies should have access to and what they can share. Some Twitter users unthinkingly post pictures of their debit cards, including their security codes, making them vulnerable to theft. Giving people access to your GPS could also make owners vulnerable to crime.

In the future contracts may be written that will allow the driver to own the vehicle, but the manufacturer owns the right to use the data collected and processed by the vehicle itself. There may be real and practical reasons to limiting how much processing power an owner may sell. A driver may overestimate how much processing power he or she can offer, robbing him or her of crucial processing power in dangerous driving situations.

Louis Renault on a car in 1903

What to make of all this? If a discussion about data mining and rented processors sounds strange in respect to connected vehicles, consider the telephone. For a long time a telephone was a device tethered to the wall. We could only imagine a cordless phone; a “videophone” was science fiction. Vehicles have come a long way. The future is an open road, and data will shift the connected vehicle forward.

The Internet of Emergency Situations: How Connected Vehicles Can Save People

Japanese ambulances  after the 2011 earthquke; with the power of the Internet of Things, it will be easier for one to add one’s own vehicle to this ambulance train. Pictrue courtesy of http://www.news-world.us/wp-content/uploads/2014/03/Ambulances-japan-earthquake-550×309.jpg

One of the benefits of connecting vehicles is the recognition that vehicles are computers on four wheels. Regardless of whether they are automated or not, this means all vehicles gather, process, and transmit information. Just as one can use smart phones to create Bluetooth-powered mesh networks through the FireChat app, we can potentially use vehicles, refrigerators, and a variety of other devices to create emergency networks to serve other purposes. Vehicles are no longer independent items on the road, but can now take part in a larger picture that can at minimum improve transportation and at most help save lives. We should take advantage of this potential by using connected vehicles and other devices to perform a variety of services to make our emergency services better than ever before.

The Internet of Things (IoT) is a powerful concept that can greatly aid humanity as a whole, whether it is by helping to automate vehicles or by acting as an advance warning system (AWS) for certain diseases, but it has a critical flaw: it requires access to the Internet to work properly. For instance, vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication are impossible without a continuous and resilient connection to the Internet to communicate location, help discover other objects that are also connected to the Internet, and perform a variety of other tasks such as broadcasting the vehicle’s future plans and whether it needs repairs. If one cannot access a satellite, then one cannot use the global positioning system (GPS) that has also proved critical for modern transportation. Such a loss is not catastrophic; humans drove without access to networking technologies for decades, and the autonomous vehicle does not require the Internet to function either. Ernst Dickmanns led a team that built an autonomous vehicle capable of driving itself in 1986, before networking technologies were widespread. However, connected vehicles, regardless of their state of automation, can drive more effectively and safely if they can share information and coordinate with other connected devices. It is also true that other smart devices can work more effectively if they are connected. Smart refrigerators cannot help their owners keep track of their contents without the Internet, heart devices cannot broadcast the status of the people they are monitoring, and disaster could strike if an oil drill loses contact with its controllers. The Internet is too important at this point to leave without backups or aid, and connected devices in general and connected vehicles in particular can help solve this problem.

The recognition that connected vehicles can receive and transmit data opens the door to using connected vehicles as emergency network broadcasters. Dr. Yuji Inoue of the Toyota InfoTechnology Center has spoken in the past about using vehicles as a hub for creating an emergency communications network (note: this is a PowerPoint file). This system can help create ad hoc networks in situations where the main providers have been disrupted. Vehicles in particular can act as mobile Internet connection points or as a sender or recipient of information requests while evacuating. For example, if an earthquake strikes and breaks down the main Internet broadcasters, evacuating vehicles could provide an ad hoc emergency communications mechanism by transmitting data from one vehicle to the next in an evacuation train. The vehicles can also serve to amplify any remaining signal and help the emergency communications network connect to the greater Internet while the connection is down. This capability can be used to boost the cloud in emergency situations (as well as non-emergency situations).

Flooding in Venice, Louisiana after Hurricane Katrina

Connected vehicles can be quickly impressed into emergency service. Just as people in the United States can join the United States Coast Guard Auxiliary to help the U.S. Coast Guard whenever they can, people could register their vehicles and other connected devices with local emergency services. In case of a crisis, the local government could send out a request to connected devices to quickly ask for aid. In an autonomous environment, autonomous vehicles can be quickly impressed into service as soon as they receive the request. These vehicles can evacuate, transport disaster survivors, supplies, and act as make-shift ambulances. One should also remember that unmanned autonomous vehicles can go to places that are toxic to humans but not to vehicles. For example, vehicles can be sent to places with toxic fumes following a volcanic eruption or areas prone to mudslides.

Finally, connected vehicles can act as inventory management tools. Cameras and other such devices could be added to the vehicles to better track what they are carrying. When. When emergency responders see a shortage of fresh water, blood, and other materials, they can recall the vehicles that are inside a disaster zone and restock them. This will allow them to create a persistent supply of goods necessary to survive. They can send out fully stocked vehicles, recall the ones that will soon be emptied, and send out new ones just as the empty ones are about to leave. These vehicles can also transport Internet broadcasters that can temporarily reconnect the area during the times when the vehicles are not there and the Internet is still down as well.

One can see from the above that the IoT and automation can greatly benefit emergency services. Just as vehicles can carry sensors that can measure how much salt is on the ground, they can also act as mobile communication network broadcasters to ensure that services that require the Internet to function well can continue to function. They can be quick aids to sudden situations, whether it is by automatically adding extra devices to emergency services (while allowing people to use their connected vehicles in normal situations such as personal driving and loaning out their cars through ride-sharing programs) and ensuring everything goes where it is needed. We know the IoT is a boon to businesses, since it allows them to make existing services more efficient, makes it easier to track when an item needs to be repaired or replaced, and gives some goods new capabilities they never had before. As the world continues to innovate, technology will not just improve our lifestyles, but also our ability to help others. Once the IoT truly gets going, it has great potential in helping the world become safer than ever before.

Autopilot: How the Partial Automation of Airplanes Can Inform On-Road Vehicular Automation

A picture from the 2017 Paris Air Show; picture courtesy of https://upload.wikimedia.org/wikipedia/commons/c/c5/Paris_Air_Show_2017_Diamond_DART-450.jpg

On June 19, the 52nd Paris Air Show began. This air show showcases technological innovations and acts as a place where airplane manufacturers showcase their newest planes. This year, a few key changes are being highlighted: how aircraft manufacturers are shifting away from focusing on one-time sales of aircraft to services such as using “Power-by-Hour” contracts and engine leases. One item that is not being highlighted is the automation of the aircraft itself. Airplanes are already highly automated and the civil aviation industry is continuing the process.  While on-road vehicle original equipment manufacturers (OEM) are still trying to figure out how to make vehicles autonomous, air travel has been greatly automated for decades. With all this in mind, the automotive sector would be smart to pay attention to the lessons learned by the civil aviation industry.

While it is an exaggeration to say that a plane “flies itself,” the pilot can- and often does- stop flying actively and tells the autopilot how to fly the plane after takeoff and before landing. The autopilot does not “drive” the plane necessarily; it merely absorbs the initial orders (and change in orders) of the pilot. These autonomous technologies are complemented by advance warning systems that can jolt a pilot to attention when necessary. Ground proximity warning systems (GPWS) warn pilots whether they are about to hit the ground or another obstacle. More impressively, a traffic collision avoidance system (TCAS) use vehicle-to-vehicle (V2V) communication to identify whether another aircraft is in the vicinity, if the two aircraft are destined to crash if they maintain their flight plan, communicate with the TCAS on the other plane (or planes), and then the different TCASs tell the pilots on the different planes how to avoid crashing into each other. As a result of these innovations, passenger aircraft- and other aircraft- no longer have engineers or navigators on board. This means the pilots can be content to mostly monitor the flight, take over when human interaction is necessary, and know the only two times they are always required to pay close attention is during takeoff and landing.

One should note that in-air OEMs and suppliers are looking into further automated innovation in the airspace. Boeing says it is developing its own artificial intelligence to run its computer-driven passenger planes in the future. Meanwhile the Defense Advanced Research Projects Agency (DARPA), the United States’ Defense Department’s research organization, is interested in creating robots that can co-pilot alongside a human. One should also note that airplanes created by the European OEM Airbus are already heavily automated, with each task using three different computers each that give orders and select actions based on a vote. This ensures there is at least one back up computer available in case one of the computers fails. It appears that as on-road automation is becoming a reality, airborne automation is further ahead and continuing towards an apparently natural conclusion.

A Flight Management System (FMS) Unit

While automating flight is a global phenomenon, commercial aircraft automation is regulated differently on a regional basis. In the United States, pilots are required to oversee the plane throughout the whole flight, even if the plane is on autopilot. By contrast Asian carriers require pilots to keep the autopilot on above 3,000 feet (~914 meters). Furthermore, while aircraft made by Boeing prefer that humans make the final decisions while the flight management systems (FMS) guide and assist the pilots, Airbus planes usually fly themselves unless the pilot overrides it. None of these approaches are perfect. Automating the plane can eliminate the chance of human error. If one were to extrapolate statistics from the on-road transportation industry, one should consider the National Highway Traffic Safety Administration (NHTSA) report from the mid-2000s said that 93% of accidents are due to human error. That number includes behaviors such as drinking and texting, but even if one excludes such situations, the data suggests that removing the human element from driving and by extension flying could save lives, if only the technology could get there first.

A white and red Turkish Airlines A330-300 with the undercarriages extended over a blue sky.

An Airbus A330, a heavily automated plane flown by Turkish Airlines

What can the on-road industry learn from all this? First of all, there is the debate about whether vehicles should be partially or fully automated.  Ford and Google have been arguing that one should skip the partially automated vehicle phase and go straight to full automation because people may become too trusting of an (incomplete) autopilot. As a result the “drivers” will stop paying attention to the road and will only return to active driving when they hear the advance warning system signal. The question is how much advance notice one should have. This question was discussed when a Tesla car got in trouble because one of its vehicles hit a truck while its driver was disengaged. In this case the accident escaped legal trouble because the driver had at least seven seconds to notice that he was about to crash and the autopilot gave him several warnings before he crashed. The question of when the autopilot should transfer control to a human is an important question in flight as well. Aside from the issue of how much time is needed to react, there is also the question of whether people can figure out how to react in the time allotted. Civil aviation has its own share of stories regarding warning systems and malfunctioning autopilots. For example, Air France Flight 447 crashed in 2009 partially because the pilots had grown used to highly automated airplanes and lacked the experience to fly the plane properly when the autopilot was disconnected due to weather problems. If the Tesla incident is an example of what can happen when the driver is too distracted or too trusting to heed the warnings, the Air France example suggests the need for backups for the autopilot. This may be why Ford says it will not start selling partially autonomous vehicles (AV). Instead, it says it will focus on manual vehicles (and those with driver assisted technologies) until 2021, when they promise to put their first autonomous vehicles on the market. By contrast BMW, Mercedes-Benz, and Volkswagen AG’s Audi are claiming they will sell semi-autonomous cars in 2018, with capabilities such as automated driving under a certain speed, which will warn drivers to take over with little advance notice, perhaps even 10 seconds. The Air France Flight 447 incident suggests the automotive industry should carefully consider copying the aviation industry’s reliance on multiple backup autopilot systems.

The aviation industry teaches us that regulation is imperative to safety and will remain an important issue, and there is no guarantee that countries will regulate automation the same way across the globe. If we are somewhere between fully manual and fully autonomous vehicles, that means regulators need to ascertain where the line should lie in terms of responsibility. As mentioned above Asian fliers are required to let the autopilot take over for most of the flight and the North Americans are supposed to treat their autopilot as a “flyer assist,” not a strict guide. Overall, we need to remember that autonomous vehicles are not a new, unprecedented technology. We have existing examples of autonomous technologies in aviation and there is no need to reinvent the wheel from scratch.


October 21, 2017 Edit: Picture was changed due to the original link from ABC no longer working.