01 The question

Can a small team of drones learn to find and put out fires it cannot fully see?

Manith Marapperuma-Contact

This project is a computer simulation in which autonomous drones learn, by trial and error, to search a patch of Victorian bushland, spot fires early and put them out. Every number on this page was measured from the simulation. None were typed in by hand.

Research simulation. Not an operational fire-behaviour or drone control system.

t 0 s burning 0 out 0

unburned burning burned out on its own put out by a drone drone, with its 800 m sensing ring

What you are watching. The same fire-spread rule the project uses, running in your browser. Each square is 80 m of land. Orange squares are burning; the wind blows from the north-west, so fires stretch toward the south-east. The triangles are drones. Each flies to the nearest fire it knows about and, once within 120 m, turns that fire's squares green. This is an illustration. The measured results in section 10 come from the full Python simulation.

02 What the tool does

A laboratory bench for repeatable drone experiments

A researcher picks a real area of Victoria on a map, clicks where fires start, sets the wind, decides how many drones fly and how far their cameras can see, then presses run. The simulation plays out second by second. Every statistic on the dashboard is calculated from what actually happened in that run.

The same setup, run twice, gives exactly the same answer. That is deliberate. Each run is stamped with a seed, a starting number for the random parts of the simulation, so a result can be reproduced by anyone with the same scenario file and seed. When a value cannot be computed it is reported as N/A rather than guessed.

To compare two drone strategies, the tool runs both on the same set of random scenes and reports the average, the spread, and a 95% confidence interval. It does not run a hypothesis test, and it says so in the output, so nobody can claim one strategy "significantly" beats another from this table alone.

Study areas

Five named regions of Victoria: Kinglake and the Central Highlands, the Grampians, East Gippsland, the Otway Ranges and the Alpine foothills. Each is a rectangle around a real place, roughly 5.5 by 8 km for Kinglake, used as a search box rather than a legal boundary.

Drone strategies

A policy is the decision rule that tells each drone what to do next. Three were learned by neural networks (DQN, MAPPO, MADDPG). A hand-written "greedy" pilot that flies straight to the nearest known fire is kept for training and comparison. If a learned policy file is missing, the tool labels it untrained and never reports it as a learned result.

Views and exports

A 2D map by default, with an optional 3D globe drawn from the same data. Every step is streamed live to the browser, logged, replayable, and exportable as JSON or CSV for analysis elsewhere.

03 How a study runs, start to finish

From a map click to a comparison table, in eight steps

This is the full methodology. Each step feeds the next, and the same code runs whether the drones are learning or being tested.

STEP 1
Describe the scenario
Choose the area, where fires ignite, wind speed and direction, how many drones fly, their speed, camera range and radio range.
scenario JSON + seed
STEP 2
Build the world
The land becomes a grid of 80 m squares for fire and 120 m squares for tracking what has been searched. Distances are measured on a local flat map in metres.
WildfireEnv.reset(seed)
STEP 3
Record an expert
A simple hand-written pilot flies to the nearest fire it knows about and hovers over it. Its flights are recorded as examples of good behaviour.
16 episodes, 4 drones, greedy pilot
STEP 4
Teach by imitation
A neural network learns to copy the expert's choices. Training repeats until, on test scenes it has never seen, it puts out fires on its own.
behavioural cloning
STEP 5
Improve by trial and error
Three reinforcement learning methods each refine the copied network using the team score, with a leash back to the expert so they never forget how to hover and extinguish.
DQN, MAPPO, MADDPG
STEP 6
Keep the best version
After each round of training the network is tested on three held-out scenes. The version that put out the most fires is the one saved.
holdout seeds 9000 to 9002
STEP 7
Evaluate every strategy fairly
All strategies fly the same random scenes with the same seeds, four minutes each, with no graphics running. Only the strategy name changes between runs.
scripts/eval_policies.py
STEP 8
Report without embellishment
Fires found, fires put out, area searched and team score are counted from the final state and reported per seed, with averages and intervals.
mean ± SD, 95% CI, no significance test

04 One environment, two uses

The drones learn in the same world they are later tested in

There is no separate "demo" version. The simulation that trains the drones is the exact code that runs behind the map. A drone strategy evaluated offline behaves identically on screen.

Measuring distance honestly

Latitude and longitude are kept for storing and displaying positions, but every distance, heading and grid is computed on a local flat map in metres centred on the study area. Measuring in raw degrees would distort distances, so the code never does.

Keeping time honestly

The simulation advances in fixed one-second steps. The on-screen speed control only changes how long the computer waits between steps; it never changes the physics.

05 How fire spreads in the model

A grid of burning squares, stretched by the wind

The land is a grid of 80 m squares. A burning square can set any of its eight neighbours alight. How likely that is each second depends on how fast fire would travel in that direction: fastest with the wind, slowest against it. That makes each fire an ellipse pointing downwind. A square burns for about three and a half minutes, then burns out on its own.

This is a deliberately simplified model, labelled as such throughout the tool. It is not Phoenix, SPARK or any operational fire-behaviour system. It is good enough to give the drones a moving, wind-shaped target to chase.

Speed at the head of the fire

R_head = R₀ · fuel · (1 + a · U^b)

In words: a base speed, scaled by how much fuel is on the ground, and boosted by wind speed U.

R₀: 0.35 m/s default, 0.22 in the evaluation runs
a, b: 0.18, 1.15
fuel: a flat 0.85 everywhere unless a vegetation map is supplied

Speed in other directions

R(φ) = R_head · (1 − e) / (1 − e · cos φ) e = clip((LB − 1)/(LB + 1), 0, 0.95) LB = 1 + c · U

In words: φ is the angle between "downwind" and the neighbour. Straight downwind gets the full speed; sideways and upwind get less. Stronger wind makes the ellipse longer and thinner.

c: 0.25

Whether a neighbour catches

p = 1 − exp(−R(φ) / d · dt)

In words: each second, a neighbour at distance d catches fire with a probability that rises with the spread speed toward it. The random draw uses the run's seed, so it is repeatable. A deterministic mode instead accumulates exposure and ignites the square when it reaches 1.

Burning out, and slope

T_burn = T₀ / max(fuel, 0.15), T₀ = 180 s

In words: more fuel burns longer. Hills would speed fire uphill, but slope is only applied when a real elevation map is loaded; the bundled runs treat the land as flat.

06 Drones, sensing, putting fires out, radio

Simple aircraft, honest cameras, and a strict rule for extinguishing

A drone is modelled as a moving point with a heading, a speed and a battery. Seeing a fire and putting it out are two different things, and the tool never confuses them: a detection is a report, an extinguish requires the drone to be physically over the fire.

Flying

Each second a drone turns toward its chosen heading and moves at cruise speed, limited by maximum turn rate, climb rate and speed. It cannot leave the study area. Wind does not push it around; that simplification is documented.

power = P_hover + k · speed² battery −= power · dt return to base when battery / capacity ≤ threshold

battery: 180 kJ default
cruise: 16 m/s at 120 m altitude in evaluation

Seeing

If a burning square is within 800 m, the drone notices it with a probability that rises with how intensely it is burning. Bright fires are easier to spot. There is no terrain, smoke or camera model, so nothing ever blocks the view. With no fire in range, a false alarm can be raised with a small set probability (zero in the evaluation runs).

p_see = p_det · (1 − p_miss) · clip(0.25 + 0.75 · intensity, 0, 1)

Putting a fire out

A drone extinguishes a fire only when it is within 120 m of the fire's centre, measured on the map. At that moment every burning square belonging to that fire becomes "extinguished", the fire turns green on the map, and it stops spreading. Hovering anywhere else does nothing. This is an abstract rule, not a model of water or retardant drops.

Talking to each other, and staying apart

Two drones can exchange messages when they are within radio range of each other (4.5 km in evaluation) and both have battery. A message can be dropped at random with a set probability. A "near miss" is logged when two drones come within the configured separation at similar altitude; a collision when they come within 8 m.

07 What a drone knows and can do

Each drone sees only its own patch, and shares one team score

Every second, each drone receives a list of 57 numbers describing what it knows, and chooses one of nine moves. It does not get a god's-eye view. It only knows about fires it has seen itself or been told about by radio. During training, a switch can grant the full fire list to speed up learning; it is off by default at flight time.

The 57 numbers a drone sees

0 to 10Its own position, heading, battery level, the wind, and how far through the mission it is.
11 to 19Up to three fires it currently knows about: direction, distance and intensity. Fires already put out or burned out are dropped, so the drone keeps looking for what remains.
20 to 31Up to three teammates within radio range, and where they are.
32 to 56A 5 by 5 patch showing which nearby squares have already been searched.

The nine moves

Fly in one of eight compass directions at cruise speed, or hover in place. Hovering only extinguishes a fire if the drone is already within 120 m of it.

The team score

After every second the whole team receives one score. Good outcomes add to it, bad ones subtract. Because the score is shared, each drone is rewarded for helping the team, not just itself. Putting a fire out is worth twice as much as finding one, which is what pushes the drones to finish the job rather than just report it.

score = 8 · fires found + 0.35 · squares put out + 16 · fires put out + 0.4 · new ground searched − 0.04 · energy used − 8 · collisions − 0.3 · overlap with teammates

This score is a training signal. It is not an accuracy percentage and the tool never presents it as one.

08 Learning methods

Three ways for a team to learn from one shared score

Reinforcement learning is learning by doing: a program tries a move, receives a score, and adjusts itself to score higher next time. Multi-agent reinforcement learning means several learners act at once, here four drones, and the score is shared. The hard part is that every drone's outcome depends on what the others do.

All three methods below start from the same imitation-trained network and use the same 57-number input and nine-move output. They differ in one thing: how much each drone looks at its teammates while it learns. The blue boxes run on every drone during flight. The dashed orange boxes exist only during training and are switched off afterwards.

Independent DQN

Everyone learns alone

Each drone learns a network that estimates how good each of the nine moves would be from its own view, then takes the best one. It treats its teammates as part of the scenery. Simple and fast, but from any one drone's point of view the others look like unpredictable weather, which makes learning noisy.

network: 57 → 128 → 128 → 9
explores: 5% random moves
learning rate: 1e-4
shares: nothing

MAPPO · multi-agent PPO

A coach who sees everything

Each drone has an actor, a network that turns its own view into a probability for each move. During training, a separate critic is shown all four drones' views at once and learns to judge how promising the team's situation is. That judgement tells each actor which of its recent moves helped. At flight time the critic is gone; drones act on their own view only. This is called centralised training with decentralised execution.

actor: 57 → 128 → 128 → 9
critic: 4 × 57 → 128 → 128 → 1
update: PPO with clipped steps
learning rate: actor 8e-5 · critic 2e-4

MADDPG · multi-agent DDPG

A coach who also sees the choices

Same actor per drone, but the central judge is a Q-network that scores the combination of everyone's views and everyone's chosen moves together. That lets it credit a good pairing of moves, not just a good situation. Because moves are a menu of nine rather than a smooth dial, a technique called Gumbel-Softmax lets the learning signal pass back through the choice. Slow-moving "target" copies of both networks keep training stable.

actor: 57 → 128 → 128 → 9
Q-network: 4 × (57 + 9) → 128 → 128 → 1
target blend: τ = 0.02 per update
Gumbel temp.: 0.8

	Independent DQN	MAPPO	MADDPG
What each drone runs in flight	Its own Q-network	Its own actor	Its own actor
What the judge sees in training	Nothing shared	All four drones' views	All four views and all four chosen moves
Kind of learning	Value-based: score each move, pick the best	Policy-based: nudge move probabilities, on fresh experience	Actor-critic: judge the joint choice, replay past experience
Strength	Simplest; least to go wrong	Stable updates; the clipped step limits damage from a bad batch	Credits good combinations of moves across drones
Weakness	Teammates look like random noise	Needs many fresh episodes	More moving parts; sensitive to settings

09 Training recipe

Copy the expert first, then improve without forgetting how to hover

An earlier, shorter attempt learned to fly toward fires but never learned the final hover that actually puts them out. The current recipe was built around that failure. The four steps are in order; each depends on the one before.

STEP 1
Record the expert
The hand-written greedy pilot flies with fires visible and its choices are recorded. A back-and-forth "lawnmower" search pattern was tried and rejected as the teacher: it covers ground but never teaches extinguishing.
STEP 2
Copy it, with emphasis on hovering
The network is trained to reproduce the expert's moves. Moments where the expert hovers over a fire are shown extra often, because they are rare but decisive. Copying repeats until the network puts out at least 1.5 to 2 fires on unseen test scenes.
STEP 3
Refine with reinforcement learning
DQN, MAPPO and MADDPG each start from the copied network and improve it using the team score. A penalty for drifting away from the expert, and a slowly shrinking share of expert-chosen moves mixed in, keep the hover behaviour intact.
STEP 4
Save only what tested best
After each round, the network is tested on three held-out scenes. The version that put out the most fires is saved. Nothing is written to disk that was not measured.

Training settings

drones: 4 per episode
episode length: 220 s at one step per second
episodes: 16 expert · 36 MAPPO · 36 MADDPG · 24 DQN
imitation: Adam, lr 8e-4, 12 epochs, batch 256
hardware: a laptop CPU, no graphics

Why the leash to the expert matters

Reinforcement learning can quietly abandon a rare, valuable behaviour if a simpler one scores almost as well in the short term. Here the rare behaviour is hovering at exactly the right spot. The penalty and expert mixing act as a leash: the learner may improve on the expert, but cannot wander far enough to forget it.

10 Measured results

Fires put out, out of four, on two random scenes near Kinglake

Every strategy flew the same two scenes: six drones, four fires igniting at random spots, four minutes, wind 18 km/h from the north-west, 800 m camera range, 120 m extinguish range. Two scenes is a smoke test to confirm the pipeline works, not a study; a paper-grade run would use around 30.

How to read the chart: each row is one strategy. The two filled dots are the two scenes (seeds 101 and 202). Further right is better. Hollow circles show the earlier, weaker version of each learned strategy from before the training recipe was fixed.

Fires put out per scene

learned by a neural network hand-written expert random moves (control) earlier version

Team score per scene, current versions

Strategy	Method	Put out, 101	Put out, 202	Squares out, 101	Squares out, 202	Area searched, 101	Area searched, 202	Score, 101	Score, 202

What the recipe changed. Before the current recipe, MAPPO put out 4 fires on one scene and 2 on the other, and on the second scene its score collapsed to −904, the signature of repeated collisions. DQN managed 2 and then 0. After adding hover emphasis, the expert leash and best-on-test selection, MAPPO put out all 4 fires on both scenes and DQN 3 and 4. A later re-test of MADDPG alone (eval_maddpg.json) also recorded 4 of 4 on both scenes.

Read with care. Every strategy, including random, found some fires, and every non-random one found all four; finding is the easy half. The learned strategies are currently matching the hand-written expert they were taught by, not beating it. Whether they do better than the expert in situations the expert handles badly, such as when drones can only rely on their own sightings, is the next experiment, not a result of this one.

11 Rules of evidence

Conventions built into the code so the numbers can be trusted

Numbers come from the simulation, never from a formula in the interface. Every dashboard value is computed from the live state. Anything undefined is reported as N/A.
Finding is not extinguishing. A detection is a report. A fire counts as put out only when a drone was within 120 m of it while it was burning. The two counts are always shown separately.
Untrained is labelled untrained. If a saved network is missing or unreadable, the drones move randomly and the run is marked trained = false. It is never reported as a learned result.
Real map data and simulated data are kept apart. Study-area boxes and satellite tiles are labelled real. Fire, drones and searched area are labelled simulated. Missing datasets are shown as placeholders, never invented roads or terrain.
Repeatability is tested. The same scenario, seed and strategy always give the same flight. Thirty-three automated tests across eleven files check fire spread, sensing, extinguishing, coverage, radio, collisions, metrics and run-to-run identity.
No unearned significance. Multi-scene reports print the average with its spread and a 95% interval, and mark that no significance test was run. Slogans like "coordination efficiency" with no formula behind them are not implemented.

12 Limits, next steps, glossary

What the model is not, what comes next, and the words used here

Documented simplifications

Fire spread is a simplified elliptical grid rule, not an operational fire-behaviour model.
Drones are moving points: no wind drift, no rotor physics, no airspace rules.
Cameras see through everything within range; there is no terrain, smoke or lens model.
Extinguishing is an abstract "within 120 m" rule, not a model of water or retardant.
Radio is a plain distance rule with random drops, not a real channel model.
Fuel is uniform, the ground is flat and the wind is constant unless the user supplies map layers. The 3D view draws a smooth globe, not real hills.

Proposed next experiments not yet run

Repeat the evaluation on 30 scenes so the confidence intervals mean something.
Test with the god's-eye fire list switched off, so learned drones must rely on their own sightings and radio messages, where the hand-written expert has no advantage.
Vary the number of fires, wind strength and radio drop rate to find where copying the expert stops being enough.
Add a real vegetation map and elevation data for Kinglake so fuel and slope are no longer flat.
Report how quickly each fire was found alongside how many were put out.

Glossary

Agent, drone, UAV: Used interchangeably here. One autonomous aircraft making its own decisions each second.
Policy, strategy: The decision rule that maps what a drone knows to what it does next. Here, a small neural network.
Episode, scene: One complete simulated mission from first ignition to the time limit.
Seed: A starting number for the random parts of a run. Same seed, same run.
Reward, score: The single number the team receives each second that training tries to increase.
Checkpoint: A saved copy of a trained network's weights, a .pt file.
Behavioural cloning: Training a network to imitate recorded examples of good behaviour before any trial and error.
Actor, critic: The actor chooses moves. The critic judges situations during training and is not used in flight.
Holdout: Test scenes the network never trained on, used to decide which version to keep.
Baseline, control: A simple strategy, such as random moves or the hand-written expert, that learned strategies are compared against.
Coverage: The fraction of searchable squares that some drone's camera has passed over.
Headless: Running the simulation with no map or graphics, for speed.

Backend, Python 3.11 with uv

cd backend
uv venv .venv && source .venv/bin/activate
uv pip install -e ".[dev]"
uv run pytest
uv run uvicorn app.main:app --port 8000

Frontend, then train and evaluate

cd frontend && npm install && npm run dev
cd backend
uv run python scripts/train_mappo_maddpg.py --outdir data/checkpoints
uv run python scripts/eval_policies.py --seeds 101 202