{"help": "https://canwin-datahub.ad.umanitoba.ca/data/en/api/3/action/help_show?name=package_show", "success": true, "result": {"Identifier": "10.1139/juvs-2021-0024", "PublicationYear": "2022", "Publisher": "Drone Systems and Applications", "ResourceType": "journal article", "Rights": "Creative Commons Attribution 4.0 International", "Version": "1.0", "author": null, "author_email": null, "citation": "Madison L.Harasyn, Wayne S.Chan, Emma L.Ausen, and David G.Barber. Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning. Drone Systems and Applications. 10(1): 77-96. https://doi.org/10.1139/juvs-2021-0024", "creator_user_id": "cde7b848-a882-4fc7-97c9-670417bd6b43", "descriptionType": "Abstract", "id": "54b0d7a1-8536-4d40-b1bb-daad81805f43", "isopen": false, "language": "English", "licenceType": "Open", "license_id": null, "license_title": null, "maintainer": null, "maintainer_email": null, "metadata_created": "2022-04-07T19:45:13.021227", "metadata_modified": "2025-06-05T14:19:03.354339", "name": "detect-video-deep-learning", "notes": "Aerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research proposes the use of deep learning algorithms to increase the efficiency of the marine mammal research workflow. To test the feasibility of this proposal, the existing YOLOv4 convolutional neural network model was trained to detect belugas, kayaks and motorized boats in oblique drone imagery, collected from a stationary tethered system. Automated computer-based object detection achieved the following precision and recall, respectively, for each class: beluga = 74%/72%; boat = 97%/99%; and kayak = 96%/96%. We then tested the performance of computer vision tracking of belugas and occupied watercraft in drone videos using the DeepSORT tracking algorithm, which achieved a multiple-object tracking accuracy (MOTA) ranging from 37% to 88% and multiple object tracking precision (MOTP) between 63% and 86%. Results from this research indicate that deep learning technology can detect and track features more consistently than human annotators, allowing for larger datasets to be processed within a fraction of the time while avoiding discrepancies introduced by labeling fatigue or multiple human annotators.\r\n\r\n**R\u00e9sum\u00e9** Les relev\u00e9s par imagerie a\u00e9rienne sont couramment utilis\u00e9s dans la recherche sur les mammif\u00e8res marins pour d\u00e9terminer la taille de la population, sa r\u00e9partition et l\u2019utilisation de l\u2019habitat. L\u2019analyse des photos a\u00e9riennes implique des heures d\u2019identification manuelle des individus pr\u00e9sents dans chaque image et la conversion des chiffres bruts en statistiques biologiques utilisables. Notre recherche propose l\u2019utilisation d\u2019algorithmes d\u2019apprentissage en profondeur pour augmenter l\u2019efficacit\u00e9 du flux de recherche sur les mammif\u00e8res marins. Pour mettre \u00e0 l\u2019essai la faisabilit\u00e9 de cette proposition, le mod\u00e8le de r\u00e9seau de neurones \u00e0 convolution YOLOv4 existant a \u00e9t\u00e9 entra\u00een\u00e9 pour d\u00e9tecter les b\u00e9lugas, les kayaks et les embarcations motoris\u00e9es dans des images de drones obliques, recueillies \u00e0 partir d\u2019un syst\u00e8me fixe reli\u00e9. La d\u00e9tection automatis\u00e9e d\u2019objets par ordinateur a atteint la pr\u00e9cision et le rappel suivants, respectivement, pour chaque classe : b\u00e9luga : 74 %/72 %; bateau : 97 %/99 %; kayak : 96 %/96 %. Les auteurs ont ensuite test\u00e9 la performance de poursuite au moyen de la vision par ordinateur des b\u00e9lugas et des motomarines dans des vid\u00e9os de drones \u00e0 l\u2019aide de l\u2019algorithme de poursuite DeepSORT, qui a obtenu une exactitude de poursuite des objets multiples (\u00ab MOTA \u00bb) allant de 37 \u00e0 88 % et une pr\u00e9cision de poursuite des objets multiples (\u00ab MOTP \u00bb) allant de 63 \u00e0 86 %. Les r\u00e9sultats de cette recherche indiquent que la technologie d\u2019apprentissage profond peut d\u00e9tecter et suivre les caract\u00e9ristiques plus r\u00e9guli\u00e8rement que les annotateurs humains, permettant de traiter des ensembles de donn\u00e9es plus volumineux en une fraction de temps tout en \u00e9vitant les \u00e9carts introduits par la fatigue d\u2019\u00e9tiquetage ou de multiples annotateurs humains. [Traduit par la R\u00e9daction]", "num_resources": 2, "num_tags": 6, "organization": {"id": "9e21f6b6-d13f-4ba2-a379-fd962f507071", "name": "ceos", "title": "Centre for Earth Observation Science", "type": "organization", "description": "The Centre for Earth Observation Science (CEOS) was established in 1994 with a mandate to research, preserve and communicate knowledge of Earth system processes using the technologies of Earth Observation Science. Research is multidisciplinary and collaborative seeking to understand the complex interrelationships between elements of Earth systems, and how these systems will likely respond to climate change. Although researchers have worked in many regions, the Arctic marine system has always been a unifying focus of activity.\r\n\r\nIn 2012, CEOS, along with the Greenland Climate Research Centre (GCRC, Nuuk, Greenland) and the Arctic Research Centre (ARC, Aarhus, Denmark) established the Arctic Science Partnership, thereby integrating academic and research initiatives.\r\n\r\nAreas of existing research activity are divided among key themes:\r\n\r\nArctic Anthropology/Paleoclimatology: LiDAR scanning and digital site preservation, archaeo-geophysics, permafrost degredation, lithic morphometrics, zooarchaeology, proxy studies, paleodistribution of sea ice, landscape learning, Paleo-Eskimo culture, Thule Inuit culture, ethnographic analogy, traditional knowledge, climate change and northern heritage resource management.\r\n\r\nAtmospheric Studies/Meteorology: Boundary layer, precipitation, clouds, storms and extreme weather, circulation, eddy correlations, polar vortex, climate, teleconnections, geophysical fluid dynamics, flux and energy budgets, ocean-sea ice-atmosphere interface, radiative transfer, ice albedo feedback, cloud radiative forcing, pCO2. \r\n\r\nBiogeochemistry: Organic carbon, greenhouse gases, bubbles, Ikaite, carbonate chemistry, CO2 fluxes, mercury and other trace metals, minerals, hydrocarbons, brine processes, otolith microchemistry, sediments, biomarkers. \r\n\r\nContaminants: Mercury, trace metals, PAHs, source, transport, transformation, pathways, bioaccumulations, marine ecosystems, marine chemistry. \r\nEarth Observation Science: Active and passive microwave, LiDAR, EM induction, spatial-temporal analysis, forward and inverse scattering models, complex permittivity, ocean colour, ocean surface roughness, NIR, TIR, satellite telemetry, GPS. Ice-Associated Biology: Biophysical processes, primary production; ice algae, ice microbiology, bio-optics, under-ice phytoplankton. \r\n\r\nInland Lakes and Waters: Hydrologic connectivity, watershed systems, sediment transport, nutrient transport, contaminants, landscape processes, remote sensing, freshwater-marine coupling. Marine Mammals: Seals, whales, habitat, conservation, satellite telemetry, distribution, population studies, prey behaviour, bioacoustics.\r\n\r\nModelling: Simulation of sea ice and oceanic regional processes, Nucleus for European Modelling of the Ocean (NEMO), ice-ocean modelling and interactions, hind cast simulations and projections for sea ice state and ocean variables based on CMIP5 scenarios and MIROC5 forcing, validation.\r\n\r\nOceanography: Circulation, temperature, in-flow and out-flow shelves, water dynamics, microturbulence, Beaufort Gyre, eddy correlations.\r\n\r\nSea Ice Geophysics:Thermodynamic and dynamic processes, extreme ice features and hazards, snow, ridges, polynyas.\r\n\r\nTraditional and Local Knowledge: Indigenous cultures, Inuit, Inuvialuit, oral history, toponomy, mobility and settlement, hunting, food security, sea ice use, community-based research, community-based monitoring, two ways of knowing.", "image_url": "2021-11-13-003953.952874UMLogoHORZ.jpg", "created": "2017-07-21T13:15:49.935872", "is_organization": true, "approval_status": "approved", "state": "active"}, "owner_org": "9e21f6b6-d13f-4ba2-a379-fd962f507071", "private": false, "related_datasets": [], "related_programs": ["941c8243-8acf-4a9e-af07-ded5acdc35f6"], "rightsIdentifier": "CC-BY-4.0", "rightsIdentifierScheme": "SPDX", "rightsSchemeURI": "https://spdx.org/licenses", "rightsURI": "https://spdx.org/licenses/CC-BY-4.0.html", "schemeURI": "https://www.polardata.ca/pdcinput/public/keywordlibrary", "state": "active", "subjectScheme": "Polar Data Catalogue", "theme": ["8f8cd877-b037-4b1a-b928-f86d9e093741", "98238b1c-5be8-41ad-8c6e-74cdc4f5f369", "3ec49cbb-4da6-4fe8-8d54-5b6ce03b49d9"], "title": "Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning", "type": "publication", "url": null, "version": null, "Author": [{"affiliation": "Centre for Earth Observation Science - University of Manitoba", "creatorName": "Harasyn, Madison", "email": "Madison.harasyn@umanitoba.ca", "nameIdentifier": "https://orcid.org/0000-0002-5741-6766", "nameIdentifierScheme": "ORCID", "nameType": "Personal", "schemeURI": "http://orcid.org/"}, {"affiliation": "Centre for Earth Observation Science - University of Manitoba", "creatorName": "Chan, Wayne", "email": "wayne.chan@umanitoba.ca", "nameIdentifier": "", "nameType": "Personal"}, {"affiliation": "Centre for Earth Observation Science - University of Manitoba", "creatorName": "Ausen, Emma", "email": "emma.ausen@umanitoba.ca", "nameIdentifier": "", "nameType": "Personal"}, {"affiliation": "Centre for Earth Observation Science - University of Manitoba", "creatorName": "Barber, David", "email": "david.barber@umanitoba.ca", "nameIdentifier": "0000-0001-9466-3291", "nameIdentifierScheme": "ORCID", "nameType": "Personal", "schemeURI": "http://orcid.org/"}], "groups": [{"description": "Inland water features, drainage systems and their characteristics. Examples of data you can find here include river and lake data, water quality data. \r\n\r\nIn CEOS, related research themes include biogeochemistry, Inland lakes and waters, modelling, remote sensing and technology, trace metals and contaminants.", "display_name": "Freshwater", "id": "8f8cd877-b037-4b1a-b928-f86d9e093741", "image_display_url": "https://canwin-datahub.ad.umanitoba.ca/data/uploads/group/2021-10-31-211937.658599hyinspirehydrography.svg", "name": "freshwater", "title": "Freshwater"}], "resources": [{"cache_last_updated": null, "cache_url": null, "created": "2022-04-07T19:49:13.974750", "datastore_active": false, "datastore_contains_all_records_of_source_file": false, "description": "Churchill Beluga Boat Drone Imagery related journal article published in Drone Systems and Applications.\r\nDOI: https://doi.org/10.1139/juvs-2021-0024", "format": "PDF", "hash": "", "id": "5bcbb0bc-425b-4fad-b7ff-4c8599043dcf", "last_modified": "2022-04-07T20:02:20.594051", "metadata_modified": "2023-05-18T21:05:56.975275", "mimetype": "application/pdf", "mimetype_inner": null, "name": "Detection and tracking of belugas, kayaks and motorized boats in drone video using deep learning", "package_id": "54b0d7a1-8536-4d40-b1bb-daad81805f43", "position": 0, "resCategory": "supplemental", "resource_type": null, "size": 4104522, "state": "active", "url": "https://canwin-datahub.ad.umanitoba.ca/data/dataset/54b0d7a1-8536-4d40-b1bb-daad81805f43/resource/5bcbb0bc-425b-4fad-b7ff-4c8599043dcf/download/juvs-2021-0024.pdf", "url_type": "upload"}, {"cache_last_updated": null, "cache_url": null, "created": "2023-05-18T21:05:57.006643", "datastore_active": false, "datastore_contains_all_records_of_source_file": false, "description": "Researchers at CEOS are often asked to write a field story about their work, to make their research more accessible. We decided to do something a little different for our work on applying machine learning to detecting and tracking beluga whales: we are presenting it as a comic-book style video!", "format": "", "hash": "", "id": "1cd6dbae-5c9d-440d-b29b-26c84fbc5a7c", "last_modified": null, "metadata_modified": "2023-05-18T21:05:56.975546", "mimetype": null, "mimetype_inner": null, "name": "One Beluga, Two Beluga, Three Beluga, Four: How to Count Belugas When You Run Out of Fingers and Toes", "package_id": "54b0d7a1-8536-4d40-b1bb-daad81805f43", "position": 1, "resCategory": "supplemental", "resource_type": null, "size": null, "state": "active", "url": "https://canwin-datahub.ad.umanitoba.ca/data/publication/beluga-graphic-novel/resource/58aed159-4a62-4c2b-9978-967ad5f356a6", "url_type": null}], "tags": [{"display_name": "Beluga", "id": "a9f25a89-b0ef-4d4d-993d-73f28e0d702a", "name": "Beluga", "state": "active", "vocabulary_id": null}, {"display_name": "Unmanned Aerial Vehicle", "id": "a6dc9001-e6da-4a84-bfec-2941d3ebce78", "name": "Unmanned Aerial Vehicle", "state": "active", "vocabulary_id": null}, {"display_name": "computer vision", "id": "d7270905-c420-4d19-aa9c-c6f818ab5b67", "name": "computer vision", "state": "active", "vocabulary_id": null}, {"display_name": "deep learning", "id": "87526358-2d8a-4c78-8375-38c132b53d5a", "name": "deep learning", "state": "active", "vocabulary_id": null}, {"display_name": "object detection", "id": "a3d44586-cba5-4685-b07a-2d2f16578353", "name": "object detection", "state": "active", "vocabulary_id": null}, {"display_name": "object tracking", "id": "28ce0864-2ed8-43d1-b80d-d79684cac63f", "name": "object tracking", "state": "active", "vocabulary_id": null}], "relationships_as_subject": [], "relationships_as_object": []}}