Got research? Get a drone

Drones expand boundaries for Binghamton scientists

For research on campus safety, Chengbin Deng, associate professor of geography, used a drone to take thousands of aerial images of campus at night, then put them together in a mosaic.

By Diana Bean '81

September 24, 2019

Ten years ago, taking an aerial photo of the ��Ƶ campus meant hiring an airplane and hoping for sunshine. The photos were pretty and showed why the main road around campus is called “the brain.”

Two years ago, Chengbin Deng, associate professor of geography, used a drone to take thousands of aerial images of campus over two nights and then used a computer program to turn the images into a mosaic.

Still a pretty picture, but it’s more than that.

It’s an example of how drones are inspiring new ways of looking at old problems and finding new challenges to be solved. To be clear, the drones — or unmanned aerial vehicles (UAVs) — are simply the platforms on which cameras, sensors and microprocessors can be mounted. But ��Ƶ faculty who use sensors in fields ranging from anthropology to engineering to geography to systems science all agree: Drones are game changers.

Seeing beyond the surface

The nighttime image of campus is a visual reference point for Deng, who does environmental criminology research, and two collaborators who are looking at what it takes to make a sustainable community. Specifically, they are focusing on safety.

“If we want a community to be sustainable, we have to make it safe,” says Yu Chen, associate professor of electrical and computer engineering.

Making it safe means understanding the landscape of the community and how people behave within it. For instance, the nighttime image shows where lighting is ample and where it’s lacking, so more lights or increased patrols might be warranted.

But when it comes to human behavior, things get complicated.

The most straightforward application, Chen says, is to fly a drone, mounted with a video camera, over a location that a police officer cannot easily or safely approach, and stream images to a dispatcher for review.

Consider a large outdoor concert: Suddenly the crowd shifts and turns away from the stage. Is it a fight — or are people dancing like dervishes? If a drone can fly over the crowd and transmit images in real time, then a human can decide if the people are throwing punches or clapping in rhythm.

“When something unusual happens, you might not want to dispatch police immediately, but you want to at least make someone in charge aware of what’s going on. It’s called situational awareness,” Chen says.

“A drone is one of the most important devices for us to move toward the future of smart cities. Many people think a smart city is one that has the internet everywhere. But a smart city is one in which you can collect data from the environment, analyze it and use it to make a decision instantly.”

Making drones “smart”

Of course, it’s not that easy.

Drones as a disaster response tool

When disaster strikes, can drones help people in need? Sung Hoon Chung, assistant professor of systems science and industrial engineering, is researching their role in humanitarian logistics.

Specifically, Chung is focusing on the transportation piece of the equation: In the wake of disaster, how can critical supplies like medicine, food and water be delivered to people who are cut off by damaged infrastructure or dangerous conditions?

Drones could work, but they are hampered by limited flying range and short battery life.

In one solution, a truck gets as close as possible to where supplies are needed. A drone takes a package and delivers it while the truck moves to the next drop point. The drone finds the truck, gets a new battery and sets off with another package.

“In this way, the total delivery time can be reduced and the delivery range can be expanded,” Chung says. “But there will always be a time gap when the truck and the drone have to wait for each other.”

He sees a way to close that gap by calculating a location where the truck and drone meet simultaneously, something that he says sets his research apart from Amazon’s planned drone-delivery service.

“Thinking to the future, I’m studying how we can optimize operations when trucks and drones are autonomous,” Chung says. “And can drones be recharged while the truck is moving?”

Chung first became interested in drones five years ago when he learned about different uses for them while at a NASA conference. Applying drones to humanitarian delivery is a natural extension of his expertise in something called the vehicle routing problem, he says.

That problem is one that carriers such as UPS must solve every day: How to move a particular volume of packages in trucks with a limited amount of space and do it cost-effectively.

The vehicle routing problem is an extension of the popular traveling salesman problem: If a traveling salesman has to sell a product in eight different cities, what is the best route to take to minimize travel time and cost?

“The idea is very simple, but finding the optimal solution can be extremely hard if the problem size is large,” says Chung, who teaches the problem as a one-semester graduate course. “And if the number of cities is greater than 100, it’s impossible to solve using conventional methods. The research and teaching focus lies in developing algorithms and cutting-edge heuristic methods to solve the problem effectively and efficiently.”

Drones have limited flying time based on battery size; 20 minutes on average. As the size and weight of sensors have dropped — making them better adapted to mounting on a drone — the amount of information they can collect has grown. So, a drone doing surveillance will have to send a huge amount of video data back to a server on the ground before it gets to a computer screen where a human can assess what’s going on.

To streamline this, Chen and his group are using a process called edge computing to turn the drone into a “smart device,” meaning the drone carries a small, single-board computer that can process the video data right in the air. Algorithms built to recognize aberrations in predictable behavior can immediately issue an alert that tells a human, “Hey, pay attention to this!”

Paying attention has been Timothy Faughnan’s job for the 37 years he’s spent in law enforcement on the Binghamton campus, including 10 years as chief of police. He is now associate vice president for emergency services.

“A lot of it has to do with familiarity. Police generally work the same shift in the same patrol zone and, over time, learn what is normal and what is not — like who parks when and where at a building. It becomes easy to recognize a pattern,” he says.

So, when Chen wanted to create artificial intelligence to recognize patterns particular to a campus community, he asked Faughnan for help.

Faughnan says he remembers asking Chen, “What role am I really playing here? And he [Chen] said, ‘It’s like left brain, right brain. We need each other. You provide real-world stuff and I provide technical stuff.”

They started with basic observation skills right from the police manual. “We trained artificial intelligence how to think like a new officer,” Faughnan says.

But what does a veteran police officer bring to the equation? Chen and a graduate assistant asked Faughnan a lot of questions, like “Why would you investigate this but not that?”

“They were taking what I was telling them and thinking in terms of what arguments AI is going to make to determine, ‘If I see this, do I do that or not do that?’

“And remember, nobody is ‘seeing’ anything,” he says.

The algorithms recognize movements and patterns, not individuals.

Protecting privacy

As important as knowing what to look at is knowing what not to look at. Privacy is a vital concern for the team.

Chen is working on a way to keep drones from turning into peeping toms.

“If you fly a drone close to windows, there will be some algorithm that says, ‘This is a window, and do not remain to look at it,’” Chen says. “Or when there is someone in their backyard, maybe sunbathing nude, we can denature the data before the image is sent back.”

It will be imperative that policymakers, stakeholders and communities understand the upside and downside of drones, says Deng. The technology will eventually change our lives, and will require policies.

“We don’t want to monitor everyone. We want to make sure we have a safe campus without violating people’s privacy,” Deng says.

Natalia Romanzo, a master’s student in sustainable communities, is wearing about 40 pounds of equipment, including a magnetometer and a battery pack. The equipment is still being used, but drones are increasingly providing a more practical alternative. Image Credit: Jonathan Cohen.

Remote sensing in remote locations

Blooms of toxic cyanobacteria — commonly called blue-green algae — have been on the rise in New York lakes, where they can contaminate drinking water, poison animals and keep tourists away.

But recognizing which algal bloom is harmful and which is not can’t be done just by looking at it.

That’s why Kenneth Chiu, associate professor of computer science, and Timothy de Smet, research assistant professor of geological sciences, along with other researchers, visited several large New York lakes with their drones. The one in the air was mounted with a hyperspectral sensor, made and donated by Corning, which can sense about 150 discrete narrow wavelength bands (human eyes sense three: red, blue and green.) The drone in the water was an autonomous boat, which collected water samples. The boat is one of two purchased with a grant from the United States Geological Survey.

The goal is to develop a machine-learning algorithm that will detect harmful algae from drone imagery alone. To do this, the algorithm will be “trained” by giving it both measured harmful algae content from water samples and the hyperspectral images. The algorithm will thus “learn” how the images relate to harmful algae content.

“When someone says the lake looks icky, is it the harmful kind [of algae]? Do they mean the whole lake or just one spot? And how much of it is there?

“To do science we really need to get a sense of it,” Chiu says. “I won’t say it would be impossible to do without the drone, but without the drone it wouldn’t have happened.”

Looking for methane wells

When asked about the impact of drones on his work in remote sensing, de Smet paused and then replied, “It’s incredible.”

First, he described how data is collected on the ground: You wear a giant battery pack and carry a heavy magnetometer. Sometimes two people are needed, one to pull a cart over the rough terrain or carry a pole with a counterweight on the back, like a boom, to keep the sensors from getting too close to the electronics.

“Not only is it antiquated, but it’s expensive,” he says. “Now the sensors are 30 grams, the size of my thumbnail. And I can put one on a drone.”

De Smet is just three years out of grad school and, already, drones have opened up his field. “Remote sensing is a career where you can do anything you want,” he says.

This summer, de Smet, Assistant Professor Alex Nikulin and student Natalia Romanzo went to a Department of Environmental Conservation site in Cattaraugus County to search for old methane wells.

There are an estimated 320,000 orphan methane wells in the United States, he says. To protect the environment from leaking methane, they should be found and capped. In New York state, that project is estimated to cost hundreds of millions of dollars — but that’s using the old technology.

De Smet hopes that using infrared heatdetecting sensors on drones will aid in the development of algorithms to help locate the unprotected wells.

“People are walking around in the forest with equipment that is heavy and awkward, doing a hunt-and-peck method. Why would you do that when you can fly a drone in a grid? You tell it the size of the grid, how many passes to make, where to land.

“We’re all going to get fat because we’re sitting in the truck,” he adds, laughing.

Posted in:

������Ƶ