Newsletter #3 - SPIRIT PROTOTYPE 2.5

The SPIRIT consortium is indeed thrilled in announcing that its technology partners have managed to squeeze yet an additional evaluation cycle between Year 2 and Year 3 SPIRIT tools prototype environments. The community of end users have already received version Y2b (alias v2.5). This is an intermediate step before delivering the final system backbone. This decision came to resonate with a three months project extension that has been requested (final approval is imminent) to challenge whether we could have covid-19 free ‘showcasing events’.

Following the delivery of:

This M30 Year 2.5 Prototype extends Year 2 functionalities and features an improved end users experience. Moreover v2.5 strives and corrects performance efficiency issues, workflow technical hiccups and several end users’ driven operational requests.

In March 2021, SPIRIT technical partners delivered to the LEAs community (i.e. Hellenic Border Police GR, Thames Valley Police UK, West Midlands Police UK and STAD Antwerp Police BE) a working prototype that indeed aspires to quality TRL6 specifications.

Our next steps would be to upgrade the system with a final touch of Machine Learning Identity Resolution algorithms as part of a set of tools aimed to provide consultation to the LEA analysts..

Stay in touch as we tread towards the end of an exciting three years journey!!

Costas Davarakis,

SPIRIT Technical Coordinator (This email address is being protected from spambots. You need JavaScript enabled to view it.)

Refined Search

It is intended that the Refined Search is to be used when the investigator has already gained some knowledge about the target person. The actual search is undertaken in two stages:

1. Level 1; this search returns a list of web pages that you would expect to see from a Google search. The investigator should read the brief overview on the page and decide whether to include the page. If unsure, the page link can be clicked to open the web page in a new tab and view its content.

2. Level 2; this search uses all of the selected pages from the Level 1 search and uses any page links found to return another list of pages. The investigator goes through the same selection process as discussed above to select the required entries.

These two processes can take some time to complete but it is important that the investigator only selects the pages that are relevant to reduce the non-essential nodes that will be displayed in the graph. Once these two processes have been completed, all of the web pages will have the main text scraped and processed to label words and phrases (entities) that are relevant to an investigator. The entities are then linked based on a set of rules that have been devised in-house to produce a Relationship List that can be displayed through the UI:

The investigator can validate each entry be clicking on the document url and view the source web page. On looking and assessing the graphical output, should the investigator feel that there is a possibility that two or more of the persons be the same person, the Relationship List is used by the Identity Resolution processes to make that assessment.

The results of the refined search are displayed in a graph

The entities and their links can be manipulated as illustrated above. All manipulations are recorded in the database and added to the Relationship List for further analysis if needed. There is a “Development” mode within these processes which is linked to a set of fake web pages and can be used to demonstrate the functionality to a live audience.

Progress Monitoring

The investigator can validate each entry be clicking on the document url and view the source web page. On looking and assessing the graphical output, should the investigator feel that there is a possibility that two or more of the persons be the same person, the Relationship List is used by the Identity Resolution processes to make the SPIRIT generates results by orchestrating microservices according to proper chains of execution: SPIRIT services do not usually run in parallel since services may ‘consume’ information ‘produced’ by other services. For example, the Natural Language Processing service waits for contents extracted by the Scraper service from Web pages which, in turn, are downloaded by the Crawler service.

The Progress Monitoring Service tracks the status of a running investigation as well as of its corresponding execution chain. As soon as an investigation starts, each SPIRIT microservice communicates the status of its activities by notifying the Progress Monitoring Service about the number of consumed and produced items (such as Web pages, texts, images or videos). Messages are managed to monitor the number of consumed and produced items for each service as well as to infer the status of each service: the service status can be (i) running if the service is sending keep alive messages or (ii) completed if the service is no more communicating with the Progressing service. Consequently, a scheduled investigation is completed if all services in its execution chain are no more running.

The Progress Monitoring Service is a microservice written in Typescript by the Lutech Team. All SPIRIT service were integrated by adopting the architectural pattern adopted in SPIRIT where: (i) other Microservices communicate with the Progress Monitoring Service by sending messages over RabbitMQ and (ii) a REST API is used to allow the UI to collect information about investigations and microservices.

Dissemination events:

- Article publication of SPIRIT project in EU research magazine.

Newsletter #4 - LAW ENFORCEMENT AGENCIES' TOOLS AND METHODOLOGY SKILLS TO SUPPORT THE FIGHT AGAINST ORGANISED CRIME

This is another very exciting SPIRIT Newsletter indeed! We are treading our mid-summer steps towards shoring our three years project safely and effectively by the end of October 2021. We have chosen to pass the floor to our hands-on intelligence investigators who will harness SPIRIT’s outcomes beyond the life of this Horizon 2020 contract. To that end we spoke to Police Captain Theodoros Gylos from the Border Protection Division of Hellenic Police, about the challenges that Law Enforcement Agencies are facing in the fight against organised crime and terrorism and how the SPIRIT tools can help address these challenges.

Matching the above Interview, we have chosen to shed light to issues that relate to how SPIRIT may provide law enforcement agencies tools and methodology skills towards supporting the fight against organised crime.

Today, law enforcement is overcoming the deficiencies of traditional data mining by turning to entity/identity resolution technology. This technology could work through SPIRIT to:

- Integrate records from many sources

- Resolve conflicts between records

- Correct errors and complete missing field

With identity matching capabilities, law enforcement agencies can identify and match individuals, groups, events, and other critical data accurately and quickly across systems, regardless of language, structure, format, location, duplication, omissions, or errors. Identity matching enables law enforcement to gather, correlate, match, and share disparate data in ways not previously possible; this is what our partners London Metropolitan University with the assistance of AES Solutions UK and the overall SPIRIT ‘red thread’ technology team are describing in this Newsletter. Enjoy!

See you soon in our next Newsletter! Have a wonderful safe Summer and always keep an eye to the Greek Letters (especially Delta ? )

Costas Davarakis,

SPIRIT Technical Coordinator (This email address is being protected from spambots. You need JavaScript enabled to view it.)

LAW ENFORCEMENT AGENCIES VIEW ON ICT TOOLS ΤΟ SUPPORT THE FIGHT AGAINST ORGANISED CRIME

Interview with Theodoros Gylos from the Hellenic Police

What are the main threats organised crime and terrorist groups pose to the International security?

In our days, transnational organized crime and terrorism undermine and international security, posing significant threats for public safety, public health, democratic institutions, and economic stability globally. Specifically, organized criminal groups manage to penetrate the state apparatus, intensifying corruption, influencing political areas, undermining the rule of law, free press, subverting the principals of the judicial systems and transparency. Furthermore, organized crime is a threat to democracy and social cohesion, raising concerns about state security, citizens’ security and conformity on government institutions. Finally, organized crime represents a significant threat to economic growth and stability, through its subversion, exploi­tation, and distortion of legitimate markets and economic activity.

How SPIRIT may provide law enforcement agencies tools and methodology skills towards supporting the fight against organised crime?

Modernization of crime methods and techniques and invention of new modi operandi, used by the criminal organized groups, makes even more necessary the confrontation of the organized crime and the dismantling of organized criminal groups. Therefore, Law Enforcement Authorities must make use of technological innovation and develop new investigative measure to counter the threat of organized crime.

Research Innovation presented in SPIRIT tools are being guided and validated by a set of relevant use cases (scenarios) proposed and monitored by partner Law Enforcement Agencies; what may be highlighted as added value in the light of the deployed use cases?

Presenting an analysis of a case study drawn from a real life setting, provides the opportunity to check tools operability, propose improvements and help Law Enforcement Authorities to become more effective and better counter criminal threats emerging from serious and organizes crime. Copping with big volume of data and diversity of information, makes data quality assessment essential. In real world data through scenarios inconsistencies and errors may be revealed and test data quality.

SPIRIT has pioneered a focused activity relating to ethics and privacy preservation. What would you consider as most important elements relating to ethics and personal data privacy (e.g. GDPR)? May we specify criteria for specific use cases, to enable intelligence investigation jobs whilst maintaining privacy? In SPIRIT by connecting entities to data spaces, how would you assess privacy preservation?

Intelligence investigation may reveal various links among entities, generating valuable data for the Law Enforcement Authorities, but also raising at the same time privacy concerns. Firstly, law provisions (e.g. Data Privacy Principle) ensure data protection when accessing, using and collecting information. Moreover, on a technical level, safeguarding data from unauthorized or accidental access or loss, may ensure data privacy. Randomization, k-anonymity and distributed privacy preserving, are proposed techniques, ensuring privacy-preserving data.

Towards a SPIRIT toolkit for Resolving Identities

By the SPIRIT ‘red thread’ technology team

An Identity Resolution toolkit, developed in SPIRIT, can help Law Enforcement Agencies (LEAs) to fill gaps and make ends meet in the sleuth process of addressing diverse organized crime threats. This service alleviates LEAs to identify and match individuals, groups, events, and other critical data accurately and quickly across systems, regardless of language, structure, format, location, duplication, omissions, or errors. Identity matching enables law enforcement to gather, correlate, match, and share disparate data in ways not previously possible.

This Identity Resolution service is derived from the SPIRIT prototype toolkit which provides the ability to recognize an entity, be it a person, place, or thing, along with associated relationships, consistently and accurately based on a Relationship List which is coming from a refined keyword search, where the LEA investigator exploits already existing overt and covert knowledge regarding candidate perpetrators.

In this job, Identity models and behavior patterns are being developed to enable the capability to resolve identities. This is accomplished through machine learning techniques that assess individuals, their roles, skills, communication channels among other elements. A SPIRIT designed identity resolution method enables clear connections and marking of possible single person duplicate identities, for the candidate perpetrators. This is exhibited in the SPIRIT prototype with links between people and entities while using relationship lists which are based on results coming out from another SPIRIT toolkit service, the Refined Search service.

After completion of the identity resolution service, a list of suspected similar identities is produced:

These entities are then linked, based on machine learning techniques, to produce a new relationship list linking suspected similar identities to be displayed through the SPIRIT presentation interface (UI):

The overall results of an Identity resolution service job are displayed in a graph, providing a holistic view of all identity data, across touchpoints, with the goal of referencing points accurately for up-to-date perpetrator identities to be determined (see nodes "Brooks" and "Mariett Snehh"):

Newsletter #2 - SPIRIT PROTOTYPE 2

SPIRIT has entered the final stage of development. Now is the time for all partners to make the extra mile and strive to render superior excellence, in this very demanding and yet simple in its perception research and innovation project. A word on recent achievements:

Following the delivery of:

a rapid end users’ prototype (M12) (Year 1 Prototype, July 2019)

a proof of concept prototype (M18) (an updated Year 1 Prototype deployed to end users, March 2020);

the foundational baseline system (M24) Year 2 Prototype, involving new image related services has been deployed to LEA premises after a period of consolidation. To that end, in December 2020, SPIRIT delivered to LEAs the Year 2 Prototype, successfully deploying it to: Thames Valley Police UK, West Midlands Police UK and STAD Antwerp Police BE. In following, within January 2021, the system was deployed to the Hellenic Border Police GR. Evaluation and assessment of the current build is ongoing and the LEAs remain committed to working in partnership with technical partners to maximise SPIRIT tools capability and to scope further developmental opportunities to enhance and refine the end user experience. Deployment to other partner LEAs (Polish Police Academy and the Serbian Police) are still facing some practical and legal issues. Nonetheless an Early Adopters Forum has been established, where the SPIRIT-Tools platform will be presented for evaluation. In the upcoming period SPIRIT will be upgrading the system to the final system backbone. (M30), to involve new services employing machine learning techniques to act as consultation tools suggesting stronger evidential links among extracted social graphs objects. We will inform you of this exciting evolution in the next Newsletter bulletin.

Stay Safe wherever you are!!

Costas Davarakis,

SPIRIT Technical Coordinator (This email address is being protected from spambots. You need JavaScript enabled to view it.)

Face Recognition Tools

One of the major new functionalities was the introduction of face recognitions tools into the SPRIIT platform. Investigators can enable and apply the new tools to any data source currently available in the SPIRIT system. Due to a unified data processing approach the tools can easily be applied to other new data sources that might be integrated into SPIRIT in the future as well. Prototype 2 already introduces such new data source by now allowing investigators to upload media data (e.g. face image) and include into the data set under investigation. The face recognition process itself is divided into two stages. The first stage is what we call face extraction. When ingesting new data into the system, e.g. by issuing a crawling, a refined search task or a file upload, SPIRIT runs the extraction on that data and detects the face region in images and classifies the age and gender. The second stage is the actual face matching phase. Here the system provides fine grained (down to the face level) selection of the query face as well as the set of faces to match against the query face(s). SPIRIT implements the face recognition tool with privacy and ethical constraints in mind. The tools ensure that no data is stored that is unlikely to contribute to the case (e.g. have a low matching similarity value). In addition, the face recognition tools do not automatically draw any conclusion based on the results. Each step (data selection, face matching selection, face matching) is designed with a human in the loop who needs to make the final assumptions.

Early Adopters

What does this mean. The 2nd year prototype is now with our Policing partners and is currently being evaluated with a view to eliciting comments and suggestions for further improvement. At this stage we are inviting Police and related agencies to be a part of our Early Adopter community; to receive the SPIRITplatform, utilise its functionality and provide suitable feedback for our Research and Developers. We already welcome two Early Adopter partners: the Belgium Police Zone Rupel and the Ministry of the Interior of Croatia. Once all administration issues have been finalised, they should be receiving an enhanced version of the year 2 prototype early February 2021.

Some events in which SPIRIT was present:

- SPIRIT attended the CoU - Workshop on Research Data in FCT-Workshop on legal and ethical clarifications- VC, on June 2020. Webinar.

- SPIRIT attended the CoU - FCT Workshop on Organised Crime and Cybercrime: Synergies, Complementarities, Platforms, on June 2020. Webinar

- SPIRIT presented the second Year prototype platform at the UIA ‘Be Feel Secure’ Workshop Organised by The City of Piraeus and the Hellenic Police on July 15th, 2020. Conventional Workshop.

- Michael Phillips, Hassan B. Kazemian from London Metropolitan University, partners of SPIRIT project and Mohammad Hossein Amirhosseini from University of East London, presented the paper "A Rule and Graph-Based Approach for Targeted Identity Resolution on Policing Data", at the 2020 IEEE Symposium Series on Computational Intelligence (SSCI), that took place in Canberra, Australia.

Newsletter #1 - SPIRIT PROTOTYPE 1

The SPIRIT project is taking a disruptive approach in the development, testing, training and evaluation of (and on) a novel system prototype, determining scalable privacy preserving intelligence analysis tools to resolve identities. The first version (v1.0) of the SPIRIT prototype, is focusing on use cases provided by six (6) SPIRIT LEAs, tested on anonymised data sets combined with developed functionality to securely trawl through open source data, behind each partner Forces’ firewall and security systems.

In October 2019, the SPIRIT prototype was successfully deployed to the Hellenic Border Police, followed by deployment in February 2020 to Thames Valley Police; additional deployment to at least four (4) LEAs has been delayed, pending either technical infrastructure or ethical restrictions at partner Forces’ sites. Basic evaluation and assessment of the current build is ongoing and the LEAs remain committed to working in partnership with technical partners to maximise SPIRIT tool capability and to scope further developmental opportunities to enhance and refine the end user experience

Integrated Ethics & Privacy Protection

SPIRIT PROTOTYPE 1 has been developed in strict compliance with the relevant ethical and legal guidelines, provisions, procedures and protocols that have been identified. The SPIRIT Consortium has followed a regulatory model with internal and external controls. The Ethical lead partner has worked closely with the SPIRIT DPO and the Ethical Advisory Board to comply with the requirements set by the Ethical Panel of the European Commission. A SPIRIT toolkit based on the implementation of a dynamic Data Protection Impact Assessment, Incidental Risks, Incidental Findings Policy, regular ethical audits, algorithm training and algorithm auditing has been put in place. These metrics and procedures to ensure the protection of citizens’ fundamental rights have been collected into the SPIRIT Handbook for legal and ethical compliance.

Keyword based refined and automated search

The Enhanced Refined Search (ERS) permits the user to define an increased set of parameters relating to the search and then further refine the initial results by selecting those that are appropriate for the investigation and should be further explored. Once satisfied with the initial results selection list the urls are available to the crawler module. The results of the searching are scraped to extract the relevant text based on examination of the html tags. This text is parsed by an open source natural language processing tool; spaCy producing a range of output to include lists of entities and noun chunks. The Automated Search is a third party/social media data acquisition service that provides information based on third party and social media APis is provided by SPIRIT. Twitter API and Google Search API have been integrated in order to download texts, images and videos to be processed by SPIRIT's analysis tools. A search can be performed starting from a twitter ID, a keyword or a list of keywords..

Content Database System

The data collected and used by the other SPIRIT tools is managed in a graph database system that has been designed for the purposes of SPIRIT PROTOTYPE 1. This system consists of three components: 1) The SPIRIT content database that contains data about investigations, about the media files discovered and used during the investigations, and about the social graph. Logically, this database is represented using the Property Graph data model. 2) The database management system used to store the content database, for which SPIRIT employs ArangoDB which is a scalable multi- model NoSQL system that supports a suitable graph database model, as well as desirable transactional properties (ACID). 3) The mediator service that provides a GraphQL API to enable the other SPIRIT tools to access the content database (queries and updates). This mediator service has been developed to provide the basis to add further abstraction layers to the graph database system. Such abstraction layers i) capture the semantics of the data via an ontology (to enable semantics-based graph analysis and sense making) and ii) include computational logic (graph traversal & analytics algorithms).

Multi purpose web crawler

A web crawler has been integrated in SPIRIT PROTOTYPE 1. The crawler adopts a master-slave architecture in order to manage incoming requests for downloading texts, images and videos from one or more Web sites. An artificial dataset was created for testing the implemented crawler. A well-known open research dataset (in the area of Named Entity Recognition) was used to generate contents of the artificial Web sites. In order to not refer real identities, all the entities of the dataset were replaced with entities from the Valcri dataset or with other dummy/fictional entities.

Graph visualization

SPIRIT PROTOTYPE 1, retrieves all data from the SPIRIT services and allows to develop interactive visualisation to see relationships and connections within their data set significantly faster. This visualization is achieved by using nodes and edges. A node represents a single data point, such as a person, a location, while an edge represents a connection between two nodes, such as a communication

Some events in which SPIRIT was present:

- TERECOM-2018 and AICOL-2018. JURIX-2018, the 31st international conference on Legal Knowledge and Information Systems, Groningen, Netherlands. Paper presented: Minimisation of Incidental Findings, and Residual Risks for Security Compliance: the SPIRIT Project

- CoU Event, in Thematic Group 4 - Cyber: Crime and Security, Brussels March 28-29th, 2019.

- Annual School and Expertise Network Day of London Metropolitan University.

- Int. Workshop on Graph Data Management Experiences & Systems (GRADES), that rook place during the ACM SIGMOD 2019 conference in Amsterdam. Paper presented: Defining Schemas for Property Graphs by using the GraphQL Schema Definition Language

- 14th meeting of the Community of Users on Secure, Safe and Resilient Societies, which it was a meeting between LEAs and scientifics, Brussels, 6th– 20th September 2019.

- European workshop on Data Security & Ethics, Brussels, 7th January 2020

See here the original Newsletter #1