Detecting Maritime GPS Spoofing Attacks Based on NMEA Sentence Integrity Monitoring

Abstract

Today's maritime transportation relies on global navigation satellite systems (GNSSs) for accurate navigation. The high-precision GNSS receivers on board modern vessels are often considered trustworthy. However, due to technological advances and malicious activities, this assumption is no longer always true. Numerous incidents of tampered GNSS signals have been reported. Furthermore, researchers have demonstrated that manipulations can be carried out even with inexpensive hardware and little expert knowledge, lowering the barrier for malicious attacks with far-reaching consequences. Hence, exclusive trust in GNSS is misplaced, and methods for reliable detection are urgently needed. However, many of the proposed solutions require expensive replacement of existing hardware. In this paper, therefore, we present MAritime Nmea-based Anomaly detection (MANA), a novel low-cost framework for GPS spoofing detection. MANA monitors NMEA-0183 data and advantageously combines several software-based methods. Using simulations supported by real-world experiments that generate an extensive dataset, we investigate our approach and finally evaluate its effectiveness.

1. Introduction

Commercial seagoing vessels, such as container ships, bulk carriers, and tankers, are among the most important means of transportation today. However, they are easy targets for different attacks in the domain of cyber and electromagnetic activities (CEMA), motivated by industrial espionage and economic sabotage to piracy and terrorism. Since the shipping industry as a whole is responsible for the international supply of goods, there is a serious risk of major economic and ecological damage caused by CEMA targeting this industry, which is by no means immune to such attacks. Therefore, it must be protected effectively, as recognized in the last two decades by governments and organizations placing Maritime Cyber Security on their agendas.

Navigation in shipping has been fundamentally changed by the advent of civil global navigation satellite systems (GNSSs), the best-known representative of which is the Navigational Satellite Timing and Ranging (NAVSTAR) Global Positioning System (GPS). Today's maritime systems are increasingly computer-aided and heavily depend on the availability of GNSS for accurate positioning, navigation, and timing (PNT), usually complemented by highly interconnected sensors within integrated bridge systems (IBSs) on board modern vessels. Despite significant efforts by all GNSS operating stakeholders to upgrade security or deploy new, more secure generations of systems, civil GNSSs currently do not have sufficient security measures as practically demonstrated in the case of GPS. However, cryptographically authenticated GNSS signals have also recently been shown to remain vulnerable to spoofing attacks.

The fact that GNSS satellite signals are relatively weak when being received on the Earth's surface makes them intrinsically vulnerable. Thus, an attacker can effectively carry out jamming attacks to prevent a GNSS receiver from processing legitimate signals. While jamming attacks might be easily detected by the crew, since they would cause obvious failures in navigational instruments, so-called GNSS spoofing attacks can remain undetected and, thus, be much more harmful. In a spoofing attack, an adversary generates counterfeit signals, which are difficult to distinguish from legitimate ones and cause receivers to incorrectly calculate position and/or timing. Particularly in maritime off-shore scenarios, attacks on PNT are often more difficult to detect and, thus, perhaps more threatening. Because the majority of the previous work refers to GPS, we also focus, without loss of generality, on GPS and GPS spoofing in this paper. However, other navigation satellite systems, e.g., Galileo, GLONASS, and BeiDou, are all based on the same principle of measuring time differences in signal propagations from satellites. Thus, methods used in this paper can, in general, be transferred to other satellite navigation systems.

Vessels have a long service life. Since maritime systems are strongly embedded, accordingly outdated technologies are used that are usually prone to various emerging types of CEMA attacks. Because particularly spoofing attacks can be very devastating, appropriate countermeasures are urgently needed. Several methods for detecting such attacks exist, as surveyed in the literature. However, the proposed methods cannot be applied to existing receivers without limitations in many cases. They often require expensive and cumbersome hardware upgrades or replacements that are usually evaluated as uneconomical when assessing cybersecurity risks. Taking legacy systems into account, we believe that it is essential to consider efficient software-based approaches for the detection of GPS spoofing, which can be seamlessly retrofitted into those systems. In summary, our contributions are as follows: we identify GPS spoofing detection methods from the literature that can operate on data provided by the NMEA-0183 and can be implemented at low cost; we propose a MAritime Nmea-based Anomaly detection framework (MANA) that incorporates these methods; we generate and provide an extensive dataset including diverse spoofing attacks; and we finally evaluate and compare the effectiveness of spoofing detection methods and demonstrate the potential of their combination to compensate for each other's weaknesses.

2. Maritime Systems on Board Vessels

A reliable and accurate position and time estimation is crucial for navigation and the situational picture in IBSs. Hence, for redundancy, the majority of vessels are equipped with two GNSS devices that are supplemented by a variety of additional sensors, networked in a maritime system. A typical system architecture comprises multiple GPS receivers supplying the IBS with navigational data via the IP-based Lightweight Ethernet (LWE). Because the data provided by GPS receivers cannot necessarily be assumed to be trustworthy, the derived PNT information must be verified, which can be achieved using our approach, MANA, implemented as an additional detector component in the IBS.

A. GPS Dependency of Nautical Electronics

The PNT data generated by GPS receivers are strongly involved in navigation. In addition, derived measurements, e.g., speed and heading, are used for multiple purposes in maritime systems. The electronic chart display and information system (ECDIS) integrates those measurements into a digital chart to provide a situation picture to navigators to support conning decisions. For collision avoidance, the radio-based automatic identification system (AIS) broadcasts vessels' positions and course information to other maritime entities. Radar makes use of PNT, since automatic radar plotting aid (ARPA) is illustrated relative to the vessel's location and orientation. Autopilots need PNT to calculate necessary course corrections. Since PNT data are used by many navigational aids, the impact of unreliable and manipulated information can be devastating. Recent incidents show that minimal course deviations may lead to groundings with costly global financial losses. Hence, in the case of CEMA, alternative position estimation and tracking systems are recommended.

B. NMEA-0183 and Maritime System Networks

Distribution of nautical data on board vessels nowadays relies on Ethernet (IEEE 802.3 product family) as a well-established network standard. The NMEA-0183 standard specified by the National Marine Electronics Association (NMEA) defines the earlier used transmission and encoding of nautical data via 4800-baud serial data bus interfaces. Although the original serial transmission of NMEA-0183 is a legacy technology, the ASCII-based encoding and message format of nautical data via so-called NMEA sentences is still used in modern IP-based protocols. NMEA over IP encapsulates sentences in UDP datagrams or TCP streams to distribute nautical information via uni-, multi-, or broadcasts. A more complex IP-based protocol is Lightweight Ethernet (LWE), which is standardized in IEC 61162-450. In addition to the use of NMEA sentences, LWE defines multicast groups and protocol extensions for the distribution of data files in the maritime system. Nautical devices and sensors, e.g., GNSS receivers, can be integrated into the multicast using additional network hardware, i.e., LWE gateways. Other protocols, e.g., NMEA 2000 and NMEA OneNet, use more transmission-efficient binary encoding schemes. Although not human-readable, the encoded information is almost equal to that of ASCII-based NMEA sentences. Thus, without loss of generality, we will focus on ASCII-based NMEA in our concept and implementation. With respect to GNSS, NMEA sentences moreover not only contain functional data, e.g., latitude and longitude coordinates, elevation and azimuth angles, and time but also quality information, i.e., carrier-to-noise density (C/N0), the number of visible satellites, and their IDs. Overall, the multicast distribution of GPS-related and other sensor data via NMEA sentences in the network enables system-wide monitoring. This is leveraged by our framework, MANA, in order to detect anomalies in PNT streams that are possibly caused by spoofing.

3. GPS Spoofing

GPS signals are an easy attack target. Due to the low signal strength at the Earth's surface, the signals can be effortlessly blocked. In addition, the civil GPS lacks encryption, authentication, or any further security measures to protect the signal integrity. In fact, the data structure, modulation schemes, and spreading codes are publicly available. Altogether, these peculiarities enable jamming and spoofing. In a jamming attack, adversaries try to suppress original GPS signals using artificial interference. As a result, benign signals become unrecognizable to receivers and can no longer be used for PNT. However, receivers are usually aware of whether they are subject to jamming attacks and can react accordingly. With spoofing attacks, the situation is fundamentally different. In such attacks, adversaries generate signals that mimic legitimate signals. Often, their goal is to deceive the targeted receiver without being detected so that incorrect PNT estimates are calculated.

A. Maritime GPS Spoofing

Besides a variety of cyber and CEMA threats against a vessel in the maritime context, GPS spoofing attacks represent a major threat and attract attention from researchers, practitioners, and industry. Those attacks are known to be feasible and seriously affecting maritime navigation. Recent reports also highlight the risk posed by GPS jamming and spoofing attacks. In 2017, a total of 25 ships reported the wrong position near the port of Novorossiysk, which pointed to the Gelendzhik airport. It is suspected that GPS spoofing caused this incident. In fact, a maritime environment makes the success of a spoofing attack more likely than in other domains. At sea, there are only a few landmarks to correlate GPS positions with derived information. Without a regular position input, alternative measures, such as dead reckoning, lose their accuracy over time. Therefore, smooth long-term attacks are difficult to detect. Additionally, navigators often unconsciously tend to blindly trust their devices. In the domain of maritime navigation, the risk of attacks on GNSS has also long been noticed by the International Maritime Organization (IMO). The urgent demand for adequate integrity monitoring was first recognized in 2001 within the resolution MSC.915(22), and further resolutions followed. A compact summary of the evolution of the IMO integrity concepts, such as e-Navigation, is available in the literature. The need for resilient GPS is also part of these concepts. The Department of Homeland Security provides the Resilient PNT Conformance Framework, with four levels of resilience reflecting the users' needs. In this context, our framework can be classified as a level 1 resilient PNT but offers the potential to reach level 2 by adding a complementary PNT source.

B. Attack Model and Scenarios

To spoof GPS receivers, an attacker can employ different techniques. The literature distinguishes between three types of attackers with increasing complexity. The attackers' signals range from simple unsynchronized signals to complex, near-authentic signals with even matching angles of arrival. Similarly, we define three attacker types, namely the replay attacker (AR), the meaconing attacker (AM), and the simulator attacker (AS). The simplest attacker is AR. Equipped with a single antenna, the attacker can generate arbitrary GPS signals or record original ones. These signals are then (re)played at a later point in time. Thus, the signals are usually not synchronized to real GPS signals. Furthermore, except for superimposed signal power, no specific signal takeover strategy is performed. Attacks of this category were already demonstrated two decades ago. The meaconing attacker AM can additionally receive and process genuine GPS signals and react to them with a small delay. The simplest attack this attacker can execute is meaconing, i.e., replaying authentic signals in near-real time. To actively break the lock of a receiver to the authentic satellites, the attacker can execute a jamming attack beforehand. A GPS receiver has a lock when a stable reception from a set of satellites required for PNT is established. Initially, the attacker, therefore, uses signals with a high signal strength to overpower the legitimate signals and to force the receiver to lock onto the spoofed signals. Once the attacker has obtained the lock, i.e., controls all the signals at the receiver, the spoofing power can be reduced. Meaconing can also be effective against encrypted signals, as the signal content remains unaltered. Advanced forms of meaconing allow spoofing of arbitrary positions by individually delaying each signal. However, a lock on all counterfeit signals is not guaranteed and depends on factors such as the target's speed. As a consequence, the resulting position of the victim is not always the attacker's position and to some extent unpredictable. This issue can be addressed by relaying the authentic signals over the Internet to the attackers' transmitter near the victim, as demonstrated with consumer hardware. Besides real-time signal processing, the simulator attacker AS can operate multiple antennas to match the original signal alignments. However, such a setup is costly and brings limitations in terms of antenna placement and effective range. The precise position and other motion properties of the victim have to be known, enabling highly accurate signal construction. To manipulate the victim's receiver, the attacker performs a seamless takeover by matching the legitimate signals and carefully increasing the signal strength. Real-world experiments involving a single-antenna AS were performed in a lab environment and on board a yacht.

4. GPS Spoofing Detection

The first spoofing detection considerations and techniques were already mentioned in the 1990s by MITRE. A theoretical basis was given by Warner and Johnston proposing different techniques, e.g., based on signal strengths or signal time of arrival (TOA) monitoring, to check the integrity of PNT data and to mitigate possible attacks. Following these theoretical considerations, a variety of practical implementations and improvements gradually emerged, beginning with Receiver Autonomous Integrity Monitoring (RAIM). A comprehensive overview of existing methods and techniques, as well as their complexity and effectiveness, that have been presented over the years can be found in the literature. The variety of anti-spoofing techniques can be classified according to the layer at which the countermeasure can be applied. For this purpose, we differentiate between system, hardware, firmware, and software layers: system-layer methods include symmetric encryption of the full spreading code, spread spectrum security codes, navigation message authentication, and L1/L2 power level (code phase) comparisons; hardware-layer methods include direction-of-arrival (DOA) monitoring, synthetic antenna arrays, signal strength monitoring, Doppler monitoring, and code/phase rate consistency checks; firmware-layer methods include TOA monitoring, PRN code and data bit latency, auxiliary peak tracking (APT), signal quality monitoring, and distribution analysis of correlator output; and software-layer methods include C/N0 Monitoring (CNM), Physical Cross-Check (PCC), Clock Drift Monitoring (CDM), Ephemeris Data Validation (EDV), and Pairwise Distance Monitoring (PDM). Note that software-based approaches can also be implemented in firmware, but that involves increasing complexity and costs. Because the goal of our solution is to be retrofittable with low effort and cost, we focus on the software-based approaches, which are briefly presented in the following subsection and discussed in terms of their applicability in the maritime context.

A. Software Controls and Related Work

Software methods are flexible, easy to retrofit, and, thus, also cost-effective. In the literature, there are methods for software-based spoofing detection, such as anomaly detection approaches that monitor the C/N0 or internal clock drifts. Other approaches validate ephemeris data or implement cross-checks with physical constraints. Similar to methods from other categories, the effectiveness of these approaches is diverse: advanced attackers cannot be detected by most software-based methods. However, they still provide reliable detection for replay and meaconing attackers. An effective approach is presented by Tippenhauer et al., which requires multiple (at least two) GPS receivers. It is based on the fact that the pairwise distances between multiple receivers in a static constellation are constant with regard to the respective determined position of each receiver (except for GPS inaccuracies). Because a second receiver is available on most commercial vessels, this particular approach can be easily implemented in the maritime environment, as already suggested by Zalewski. With high-quality receivers typically used in maritime systems, the inaccuracy of position determination is expected to be reasonably low, which presumably increases the detection probability of this Pairwise Distance Monitoring (PDM) approach. Zalewski applied Tippenhauer's approach to the maritime context. Using a mathematical model and simulation, he shows that it is practically not feasible to spoof multiple GPS receivers by a single transmitter so that their relative distance remains. Recently, another approach using multiple receivers was presented for the energy sector: instead of considering the effects on position, the authors focus on the time provided by GNSS to protect electrical substations, relying on NMEA sentences but limited to the PDM method. In other work, software-based approaches for anomaly detection using GPS spoofing in the context of unmanned aerial vehicles (UAVs) are explored with machine-learning one-class classifiers requiring only non-anomalous data for training, along with a shared dataset of three UAV flight recordings — however restricted to timestamped positions of a single receiver and thus inappropriate for analyses of PDM methods. In the automotive sector, a PCC method correlates GNSS position estimates with the speedometer of the vehicle, also using NMEA sentences. A spoofing detection and mitigation approach for the maritime context uses RAIM in combination with a PCC method based on a motion model of a ship in order to detect offsets to the predefined route, while the mitigation mechanism is based on a genetic algorithm — but in addition to route information, the motion model requires other sensor inputs, whereas our approach entirely relies on NMEA data provided by GPS devices and does not require route information. Similar to our approach, other authors base their spoofing detection on NMEA data but in the context of smartphones: in a physical laboratory environment, they generate spoofing signals dictating positions on a specific route, processed by a stationary GPS receiver — however, their investigation focuses solely on static scenarios, which limits general applicability. In summary, in the field of GNSS spoofing detection, several related works exist that build on software-based methods and, among them, some that obtain their data also from NMEA messages. However, these works often focus on one specific approach. Our work, in contrast, presents a holistic, modular framework that adapts and combines different methods of existing work, offering flexible configuration and tailoring them to the needs of ships in the maritime domain. The later evaluation will show that this combination is necessary to cope with the entire attack space.

B. Complementary Approaches

Complementary approaches from the area of resilient PNT that are orthogonal to our work — alternative localization technologies such as Public Land Mobile Network (PLMN) infrastructure, crowd-sourced WiFi, the maritime R-Mode (Ranging Mode) based on signals of opportunity (AIS, VDES) with up to 40% worldwide vessel coverage and 12 m real-time horizontal positioning accuracy at 95% in the Baltic Sea testbed, maritime navigation radar, and terrain navigation — represent complementary countermeasures that can be successfully combined with our approach, with great potential not only to mitigate GNSS spoofing attacks but to entirely prevent them.

5. NMEA-Based GPS Spoofing Detection Framework

Building on a versatile selection of multiple existing software approaches presented in the previous section and leveraging the availability of PNT and NMEA data in the network, our framework MANA aims to provide low-cost yet effective integrated integrity checks as a countermeasure against GPS spoofing. Moreover, the framework allows for comprehensive comparison of the selected approaches to find out if a combined solution can compensate for their individual deficiencies. MANA, which is modularly implemented in Python 3, relies on standardized NMEA sentences. Thus, it enables generic, very flexible, and easily retrofittable deployment by adding a detector component at a central position in the maritime network. To the best of our knowledge, no directly related work provides a comparable framework.

A. Concept of MANA

The concept of our framework: a stream of NMEA sentences, dispatched by at least two GPS receivers within a network, is taken as input. Alternatively, recorded network trace files can be used as input. For each detection method, a set of fields within NMEA sentences is defined that are relevant to the individual detection approach. These fields are continuously monitored such that every state change triggers the corresponding detection method(s). The relevant required information can be obtained from different and partially redundant NMEA sentences generated by GPS receivers. However, in the end, the set of required sentences can be reduced even to the three GPS-related types $GPGGA, $GPRMC, and $GPGSV. The detector generates an alarm if any of the methods indicates potential spoofing. Based on individual methods as basic building blocks, our framework allows the implementation of more complex and sophisticated detection, i.e., by composing the outputs of those methods. This offers great potential to further increase the overall detection capability; however, a detailed investigation is out of the scope of this paper. In order to evaluate if the different methods can compensate for each other's weaknesses, a simple strategy for combining different methods is nonetheless elaborated in the evaluation.

B. Methods for NMEA-Based Detection

MANA comprises five software-based detection methods from the literature, namely PCC, PDM, EDV, CDM, and CNM. With the assumption that the data are smooth or predictable under normal conditions but not in case of a spoofing attack, these methods are implemented using thresholds. Physical Cross-Check (PCC): The PCC method is subdivided with respect to the physical features considered for the respective cross-check, i.e., speed over ground (SOG) (PCCsog), rate of turn (ROT) (PCCrot), and height above sea level (PCCheight). While SOG and ROT are expected to stay within certain limits, these limits can be exceeded in the event of a perceived spoofing attack. Sudden jumps in position can result in an unreasonably high SOG and ROT if the distance or angular deviation from the course is extreme enough. To restrict the attackers' possibilities, we use maximum thresholds for PCCsog and PCCrot, which must be plausibly selected for the respective vehicle. While the SOG threshold should be set slightly higher than the maximum speed of the vehicle, more tolerance is required for the ROT threshold. The ROT is subjected to a measurement error, which is particularly high when the vessel is slowly or not moving. Therefore, it is useful to consider only ROT measurements when the SOG exceeds a certain threshold, e.g., half of SOG threshold. The altitude is expected to be close to sea level. Thus, static thresholds above and below the sea level are defined for the PCCheight method. Pairwise Distance Monitoring (PDM): PDM monitors the static formation of two receivers and requires position data simultaneously recorded for each receiver, but these are generally not synchronized. Conventional GPS receivers only update positions with a frequency of 1 Hz. Hence, the states of the two receivers have to be aligned in time. For this purpose, one of the receivers is selected as a reference. Then, the other receiver's state is linearly interpolated between its temporally adjacent values, i.e., the time, latitude, and longitude immediately before and after the corresponding reference. Moreover, we use an exponential moving average (with α = 0.1) to reduce the effects of small-scale noise in the position sequences. Once the estimates of both receivers are aligned, the measured distance between the geographic position of the receivers can be derived. A spoofing attack is detected if the distance is smaller than the minimum threshold. Ephemeris Data Validation (EDV): Satellite positions are to some extent predictable. As a data source, we use so-called two-line elements (TLEs). If the TLEs are up-to-date, the predicted satellite positions can be used to validate the estimated ones. However, the decrease of TLEs' accuracies with increasing age and the integer accuracy of NMEA sentences with respect to elevation and azimuth need to be considered. This is checked by the EDV method by defining a maximum allowed deviation for both angles. Furthermore, the elevation of all satellites needs to be above a static minimum threshold, ensuring that they are actually visible to the receiver, since implausible constellations indicate potential attacks. Clock Drift Monitoring (CDM): With regard to CDM, a linear clock drift of each device is assumed, which is reasonable for a certain interval in time. The drift of an individual device can be derived by continuously comparing the received GPS time with the local system time. If enough drift measurements are available, a drift function is derived by a linear regression providing the expected clock drift for a given point in time. The difference between expected and measured drift is then compared with the predefined threshold and, in cases where deviation of the clock drift exceeds this threshold, a potential spoofing attack is indicated. C/N0 Monitoring (CNM): This method monitors carrier noise density and derives indications of possible attacks in cases where the noise level exceeds the given threshold. An evaluation of the CNM detection method requires real-world data or a simulation that includes a realistic signal propagation model with the associated random processes, particularly C/N0. Our simulation environment, which is used for the evaluation, does, however, not include such a signal propagation model. Thus, CNM is excluded from the later evaluation.

6. Simulation Environment and Dataset

For ease of feasibility and better reproducibility, we use simulations to evaluate the effectiveness of our approach but collected real data in experimental field trials to (i) calibrate simulation models and (ii) appropriately determine thresholds for the detection algorithms, i.e., for the parametrization of MANA. For the latter, it is important to find the right balance between triggering false alarms (false positives) and missing the detection of spoofing attacks (false negatives). For this purpose, we carried out two experiments, a static and a dynamic scenario. In the first experiment, a stationary measurement was conducted to collect static GPS tracks that enable trace-based modeling of GPS errors in our later simulation. In the second experiment, we used two mobile receivers moving in a static formation, i.e., at a fixed distance to each other, and continuously collected all data provided via NMEA sentences. The goal of the second experiment was to record the clock drift and position measurements of the GPS receivers, both for modeling in the simulation and to support the determination of thresholds for clock drift and distance between GPS receivers, required for the CDM and PDM detection methods.

A. Simulation Environment

We use an in-house simulation environment for maritime networks including various sensor information, such as course over ground, heading, compass, and AIS, with a special focus on GPS and its spoofing. The simulation environment is capable of simulating multiple ships, where each ship calculates its current position and time by triangulation. Physical vessel-related parameters such as velocity and turn rate were set to be appropriate for the commercial seagoing vessels that we consider in this paper: velocity 20 kn, rate of turn 0.5°/s, two GPS receivers at 4 m distance. For a more realistic simulation, we implemented a trace-based approach to model natural GPS noise using data from the first experiment. In addition, random noise is artificially added to the time derived from GPS using a Gaussian distribution (μ = 0.0012, σ = 0.0076) and a clock drift according to the second field experiment (≈10.55 μs/s, typical for quartz oscillators). Furthermore, based on the results of the second experiment, which showed that an insufficient distance (of 2.5 m) between the receiver pair limits reliable detection, the distance was increased in the simulative evaluation to 4 m — a distance between 3 and 5 m can be considered feasible for the PDM method.

B. Modeling GPS Errors, Clock Drift, and Calibrating Thresholds

To include natural GPS noise in the simulation, we chose a trace-based approach: two GPS receivers were installed in fixed positions, and their outputs (NMEA sentences) were recorded for multiple hours. As expected, we observed an averaged error in the position estimation of each receiver of up to several meters induced by natural noise and a varying C/N0, plus a clear temporal correlation of errors for both distance and azimuth. Especially when processing algorithms average the (simulated) GPS error in time, this temporal correlation can have a crucial impact. Therefore, errors must not be randomly selected from the recorded tracks without considering their temporal correlation: recordings of static GPS receivers are used as a noise source and a sequence of offsets (distance and azimuth) to the fixed position is constructed; for each replication and each receiver, a random index is chosen as a starting position, incremented at each time step, and the corresponding offsets are added to the simulated position measurement. Concerning the internal clock drift, a slight drift was found that is similar for both receivers (roughly 50 ms during 80 min), constant and thus growing linearly over time, with variance attributable to data processing within the receiver and for logging; linear regression yields approximately 10.55 μs/s. To determine suitable thresholds balancing false positives and false negatives — tolerating occasional false negatives of individual methods since they are complemented by other methods, hence rather relaxed thresholds overall in MANA — the detection algorithm uses a threshold of ±100 ms for CDM; PDM compares measured distance against the known real distance and requires receiver spacing significantly larger than 2.5 m, since estimated distances can become very small due to measurement errors of both receivers and falsely trigger the minimum threshold.

C. MARSIM Dataset

Based on the simulation environment, we generated a dataset, hereinafter referred to as the MARitime SIMulated (MARSIM) dataset. It consists of multiple scenarios (benign and attacked) with a duration of 120 s, provided as network traffic recordings (in packet capture (pcap) format). Each recording contains the NMEA sentences transmitted within the network of a single vessel heading north from a fixed point with constant velocity. By filtering background network traffic, the recordings are reduced to two PNT transmitting devices, namely the pair of GPS receivers placed at a fixed distance of 4 m from each other. All information necessary for the detectors discussed in this work can be retrieved from these records. The system clock results from the timestamps of the entries and can thus be compared with the GNSS time within the recordings. With a probability of 50%, spoofing attacks were started after 60 s in a scenario. The attacks are executed by the attackers AR, AM, and AS: AR replays a recording with a certain age of a ship that followed a similar route as the victim but shifted eastward by a specified distance; AM performs a meaconing attack with a delay to its own static position, located relative to the victim at a distance to the east at the onset of the attack; AS constructs signals that will slowly shift the victim's position with a velocity defined by shift speed and an azimuth angle of shift angle. All three attackers are equipped with a single sending antenna — for mobile targets, a multi-antenna AS attack is very complex and extremely difficult to realize in practice; since most vessels are moreover not equipped with more than two GNSS receivers, which also cannot acquire the signals' angle of arrival, and the detection of such sophisticated attacks is not expected to be feasible using software-based approaches, we only simulate single-antenna attackers that have actually been demonstrated in practice. The mentioned parameters of the individual attackers are expected to have an impact on the detection capabilities: the two-dimensional parameter space has 19 gradations per dimension, resulting in 361 parameter configurations per attacker (AR: distance {0–36 m}, age {0–240 ms} ∪ {1 min, 1 d}; AM: distance {0–36 m}, delay {0–270 ms}; AS: shift angle {0–180°}, shift speed {0–72 kn}). A set of 20 benign and 20 spoofed recordings differing in noisy position and time measurements are created for each parameter configuration. Hence, in total, our labeled dataset consists of 43,320 scenarios and can be scientifically used for the development and benchmarking of GPS spoofing detection and prevention methods.

7. Performance Evaluation

In this section, we evaluate the effectiveness of MANA to detect GPS spoofing attacks based on the MARSIM dataset: after describing our methodology and metrics, we investigate the capability of each method of our framework and evaluate their individual potential.

A. Methodology and Metrics

For our dataset, we propose a binary classification task. Thus, for each file of the dataset, the method needs to decide whether the corresponding scenario is spoofed or not. Furthermore, we group the scenarios by the type of attacker involved, i.e., AR, AM, and AS. The performance is measured using the common metrics precision and recall: recall represents the actual detection capability, while precision can be considered as the reliability of a method's indication. We experimentally adjusted the parameters max speed and max rate of turn to minimize false detection of spoofing attacks under benign conditions (i.e., relaxed thresholds: PDM min distance 2 m, min speed 15 kn; PCC max speed 30 kn, max rate of turn 7.5°/s; CDM max clock drift dev 100 ms) and consequently achieve high precision. CDM's threshold is set based on our field observations, whereas for min distance and min speed, half of the actual distance between receivers and max speed, respectively, are used. While this decision may inherently have a negative impact on the recall, we expect that a combination of methods can compensate for the potentially lower sensitivity of each method.

B. Evaluation Results

The results of our evaluation reveal that the success of spoofing detection methods investigated in our experiments depends, in different ways, on the attackers' parameters. Some methods depend on both attack parameters (PCCsog and PCCrot), others only on a single parameter (CDM), and still others are found to be independent of the selected parameters (PDM). According to the relaxed thresholds mentioned above, the mean precision over all displayed methods is 0.92. The methods in the focus of this evaluation are PDM, PCC, EDV, and CDM. In accordance with the classification of known GPS spoofing detection techniques, we anticipate PDM to perform best and the other methods to perform slightly lower, while we suspect that the effectiveness of PCC methods varies considerably. By checking the fixed distance between receivers, PDM shows the most consistent performance over all scenarios. While there are missed detections (false negatives) for all attacker models, they do not appear to be correlated with the attackers' parameterization. Missed detections are caused by noisy position measurements that keep the calculated distance between receivers above the min distance threshold. In comparison, the rather simple PCCsog method, which detects spoofing attacks based on unrealistically high SOG estimates, gives all three attackers a fairly large margin to remain undetected. Replay and meaconing (AR and AM) are only limited to a similar extent by the choice of the distance between the victim and the recorded ship or between victim and attacker. The exceeding of this 26 m limit is almost always detected. The transition between the undetectable and the detectable area is smoother for AR than for AM, probably due to the movement of the recorded vessel that is replayed in comparison to the static position of AM. The relationship between the AS parameters and the detection capability is more complex. While an increasing shift speed in the direction of travel (i.e., shift angle of 0°) does not offer much margin for the attacker (roughly 12 kn), the margin significantly increases up to about 46 kn as soon as the shift angle passes the mark of approximately 140°. The victim interprets a shift angle above 140° and a shift speed of up to 40 kn as traveling in the opposite direction. Similar to PCCsog, PCCrot is insensitive to changes in signal age or delay caused by replay or meaconing attacks (AR and AM). However, the attackers are already exposed at a distance of roughly 12 m, leading to a significantly increased ROT measurement. The detection of the simulator attacker AS again depends on both shift speed and shift angle. A low shift speed allows for a large shift angle and vice versa. An exception has similar reasons as above: the area of missing detections is caused by the min speed requirement of PCCrot — if the shift vector composed of shift speed and shift angle points approximately in the opposite direction of travel, the total speed derived from GPS may fall below that threshold. If the min speed threshold is decreased, the caused blind spot eventually disappears. Nonetheless, the threshold needs to be sufficiently high so that the ROT measurement based on noisy GPS positions is reliable and does not cause false positives. CDM directly monitors the local system clock of the device executing MANA with respect to the time provided by GPS. This results in a limitation of at most 75 ms for the signal age of AR and the delay of AM. A time jump of this magnitude with additional noise often exceeds the allowed drift range of 100 ms. Thus, both age and delay need to stay below this mark to avoid being detected. All other parameters are time-independent and have no influence on CDM's outcome. The two methods, PCCheight and EDV, proved ineffective in our simulations: PCCheight and EDV detect AR only if the signal age exceeds several hours or minutes, respectively, whereas AM and AS were found to be hardly detectable with these methods in our scenarios.

C. Ensemble of Methods

The performance of an ensemble of all investigated methods combines the binary outputs of the individual methods with a logical OR: as long as at least one method is triggered, spoofing is detected. Note that this initial approach is rather simple and weights each method equally. The exploration of improved combinatorial methods is part of our future work. For attackers AR and AM, a combination of the methods PDM, PCCrot, and CDM is sufficient, while AS requires PCCsog in addition to PCCrot and PDM. Overall, it turns out that a significant area of the parameter space for each attacker is solely covered by the PDM method, which thus significantly contributes to the combined performance.

D. Impact of Distance Between Receivers on PDM

The performance of PDM depends on the distance between receiver pairs. To investigate the impact of this distance on the recall, we varied the distance of two receivers between 2 and 10 m with 100 increments in an AS scenario. For each increment, 100 repetitions are conducted. We keep the shift angle and shift speed fixed to zero, effectively disabling the attacker. However, the detection capabilities of PDM are not related to the parameter choice of the attacker. Hence, the only varying factor is the GPS noise. As can be seen, the distance significantly impacts the capability of PDM. The impact vanishes at a distance of roughly 6 m, which is consistent with prior results.

8. Discussion and Outlook

Simulating reality in all its details is generally too complex and not target-oriented. Therefore, our simulator comes with some reasonable abstractions, in the context of which the CNM method has also been excluded from the evaluation. Still, an investigation of CNM's detection capabilities, especially on advanced signal takeover strategies, would be desirable and is left for future work. Other random processes are simplified in the simulation for the sake of abstraction. A trace-based approach was used to model the GPS error. Further modeling of random probability distributions was chosen and parametrized based on field experiments. Our simulation equips each attacker with only a single antenna, although advanced simulator attackers AS can be in possession of multiple antennas. The effectiveness of PDM weakens when attackers use multiple antennas. However, this could be simply counteracted by adding more receivers to the victim to further reduce the dimensional freedom of the attacker. As a rule of thumb, PDM requires always one more receiver for detection than antennas available to the attacker.

Overall, the methods in our framework MANA showed promising results. While the PDM method, in particular, covers the entire attack space, PCC methods have the potential to further improve the effectiveness of MANA's detection capabilities in many scenarios. In this context, the ensemble of individual methods plays an important role. Although our simplified approach is sufficient for many scenarios, there is a limitation: all methods are weighted equally, requiring strict thresholds to avoid false positives. Therefore, many insights of relatively weak methods are discarded, decreasing the overall performance. A possible solution to this problem may be to support the ensemble with neural networks to predict a single spoofing indicator. This approach would even allow temporal information to be included in the detection process. Such a network could be trained with our dataset and later refined with short network recordings of the actual vessel in which the detector will be installed, eliminating the need to manually set thresholds.

Nevertheless, it should be noted that there will never be a complete guarantee for the detection of spoofing attacks. Therefore, for the purpose of countering CEMA-based GNSS attacks, it is necessary for practice to develop complementary, GNSS-independent localization systems such as the R-Mode and to integrate them into the localization process for reliable situational awareness. In view of the current autonomy trend and the first unmanned vessels now reaching practical operation, the need becomes more urgent. For manned ships, GNSS spoofing can be detected and even mitigated to some degree by using other navigation technologies such as radar or visual aids to verify the vessel's position. If crew members suspect that GNSS signals are being falsified, they can also manually adjust the vessel's course and inform other vessels or port authorities of any navigational problems. In contrast, for unmanned vessels, the effects of spoofing can be much more severe, as these vessels tend to rely almost entirely on GNSS for navigation and communications and generally depend more heavily on automated systems. However, they may not yet have the ability to verify their position and accordingly adjust their course. Without crew members on board, it may also take longer to detect and respond to spoofing attacks. Therefore, the risk of collisions, groundings, or other accidents is much higher.

9. Conclusions

The fact that GPS signals are weak when they reach the Earth's surface makes it easy for adversaries to attack navigation systems, which is particularly problematic in the maritime domain, which is heavily reliant on GNSS. In this context, the paper provides a brief overview of research on detecting and mitigating GPS spoofing attacks at different layers. Since a simple and cost-effective retrofit is crucial for maritime practice, we focused on software-based methods. We proposed MANA, a novel framework comprising a selection of methods that continuously monitors and analyzes information derived from GPS, delivered as NMEA sentences via the maritime network.

Through a maritime simulator that uses real-world data to model realistic behavior, a comprehensive labeled dataset was generated, including legitimate and spoofed samples. This dataset provides not only the basis for our evaluation but can also be used for future benchmarking. In our evaluation, we compared the effectiveness of MANA's individual methods and show that it greatly differs. Pairwise Distance Monitoring (PDM) that requires multiple receivers was identified to be the most promising method in our software-based approach and is particularly applicable to maritime systems. We show that PDM achieves reliable detection for a wide range of common GPS spoofing attacks while still leaving room for improvements. As our evaluation suggests, alternative methods could support PDM and thereby compensate for remaining deficiencies in many cases. In our future work, we thus plan to extend our evaluation and investigate the potential of a smart ensemble of individual detection methods of MANA.