Sharp Eyes: Mass surveillance of foreigners in China - Part 1
A publicly exposed web front reveals new details on a tracking system for foreigners and delivers many insight into the capabilities of security organs to track individuals in real time.
In our line of work, you have to be lucky sometimes. In a previous post we explained how unsecured and forgotten dashboard and web pages can reveal a lot about China's security and surveillance system. Now, we can reveal a much more detailed insight into how such a system could look like in real life.
NetAskari got exclusive access to a web front-end demonstrating a remote tracking system especially for foreigners. It is developed for the Public Security Bureau in the region of Zhangjiakou (a prefecture of Hebei province about 60 km west of Beijing).
Disclaimer: it is still a test-system and not connected to a real time data environment.
But it is filled partially with real data from many real people of foreign nationality who have resided ( or still reside ) in China. It outlines very clearly where the journey in mass surveillance has been going in China over the past years and how foreigners are definitely at the center of the states’ attention again.
Another abandoned project ?
The system is clearly meant as a demo, with many functionalities just hinted at or simulated with place holder information. Originally considered abandoned, some minor changes to the UI could be discovered during the observation time of 3 months though. Since the last changes, the system has grown increasingly unstable as the underlying REDIS service seems to struggle to run reliably. A sign that nobody seems to actively maintain the platform anymore at this stage.
After passing the login screen the user is greeted with a traditional ‘data dashboard’ in the blue style, that is so often used by Chinese cyber security dashboards. The designs always seem to take a lot of inspiration from video games or popular sci-fi movies, coming across as too flashy for what the products contains.
Focusing on the region of Zhangjiakou, a mountainous area that is often used for winter sports and thus was the location of the 2022 Olympic Winter Games, the dashboard gives us a quick run down of all foreigners registered or currently living in the area. The user can choose to track changes from year to year from 2023 until 2026.
An extensive menu on the left reveals a rich set of functions. The first interesting tab we are gong to explore is ‘Resource library’ ( ). It contains choices as ‘Key personnel’ (a euphemism often used by Chinese security services for people under active surveillance), ‘Fugitive inquiry, Permanent residents, International students and Travel information’. But is this system any real ? Is it just another "hobby project" another "student assignment” filled with fantasy data which is hard to attribute as we have seen before ?
Turns out, this time it was different. The real surprise came after clicking on the men point 记者档案查询, which translates best as ‘Inquiry for Journalist files’. This presents a complete set of all the foreign journalists based in Beijing around 2021, including passport numbers, ID photos, private cellphone numbers, dates of birth etc. The ID pictures are recognizable as the typical shots that are taken from journalists during their visa (renewal) application process at the ‘Entry/Exit Bureau’ in Beijing by official staff.
All in all, this data is in the possession of Beijing's government systems, it is not just floating around on the internet, accessible to anyone. This is another hint that this system was setup by someone with access to government resources.
None of the journalists are located in Zhangjiakou of course and this database seem to exist mainly to run queries on persons arriving in the prefecture. Chinese authorities have been tracking journalists for a long time and getting a reliable and automated pre-warning will be feature security services appreciate.
Another group that can be separately inspected is all the foreigners who work or reside inside the prefecture long-term ( long term residents ). Individuals who have regional employers and/or have their residency in this area, are intended to get tracked on a daily basis according to the data sheets we have seen.
An operator can pull up quickly travel and movement activities as well as more detailed personal information such as: their registered address, visa details, and employer status. But there is much more…
The screenshot above needs some more attention as it gives us a lot of details on what authorities are interested in when it comes to personal data. For convenience, the charts are machine translated ( so expect some janky translations, but for our purpose it will do ):
This reveals that the personal details go far beyond normal residency registration data, migration information etc. it tries to provide a very detailed profile of a resident.
Also interesting is the mention of gasoline purchase. Not exactly sure why this is treated as a separate data point but my best guess is, that it has to do with the general fear of the authorities of self immolation of protestors in the past. Than there is a general focus of movement of targets, especially during "sensitive periods” and quite a focus on the social circles ( more on this further down ).
From what we see here, it makes sense to assume, that this is not just data collected on foreigners but just a standard template applied to every citizen of the country.
Train car and seat number
Other precompiled sets of profiles are ‘fugitives, key personnel, international students’ and people who seem to have minor violations of work or residency regulations. And their movements can be closely followed as it looks like.
For example you can see travel data from people who went to the Zhangjiakou sub province via train from Beijing or Shanghai, which includes in which carriage and which seat they were sitting.
The user can retrieve more details about an individuals whereabouts as well. A quick overview presents additional data points like hotel and hospital visits, and how often the subject was tracked (via cameras) etc. Most likely this is based on AI enabled public video cameras, but more on that later. The movement detail therefore can be quite granular and is intended to be in real time.
Each individual is also assigned with a ‘risk level’, although it is unclear on the basis of which information this is created and in which context it is used. At the time of access to the system, all the subjects were given a 63% ‘risk level’, whatever that may mean. It gives the impression it is just a place holder value for now, as real live data was not available for this demo.
Real time tracing ?
People that are marked as ‘traceable’, (轨迹) get an additional window popping up, that can deliver detailed tracking of a subject in Zhangjiakou. Those earmarked 'traceable’ individuals, whether it is for illustrative purposes or based on a listing of earmarked targets from inside China's public security organizations, can be assigned to a specific person.
As you can see in the screenshot under here, ‘control’ most likely means that active tracking on the target is performed.
More data, better results
What we have seen so far, is that the system seems to need input from traditional surveillance cameras with face recognition capabilities, travel records of train, airplane and car companies, hotel registrations etc.
But what made the system more interesting to me, is that it does not only take in those “official” data streams. As Zhangjiakou is a famous winter sport area, many foreigners in Beijing go there for skiing or snowboarding. And the system seems also to take photos from the ski-pass system, that is powered by facial recognition and fed it into the tracking mechanism.
There it detected for example some of the aforementioned journalists and we see again the marking for ‘trackable’.
The aim of powerful surveillance is to feed the system a wide range of different inputs from a variety of sensors enabling the construction of a complete profile including movement data beyond the superficial visual data on street-level.
Not just on airports, train stations or government offices/services, but it can be found almost everywhere: in shops, to get paper in a toilet and even on vending machines. The above scenario shows that this systems might be eligible to contribute to the surveillance project ( something we ourselves often thought is more based on the fantasy of western commentators than rooted in facts ).
Even if track-and-trace systems get fed with only a small subset of this kind of services from all over China, this provides a powerful ability to follow a target.
Power through relationships
Tracking subjects and targets via AI face recognition is not particularly new, nor is it only a capability of Chinese security services. But the demonstrated system here goes much further than just video tracking of individuals.
The power lies in its ability of group analysis, indicated by the provision of precompiled target lists as shown above and involving other sources to cross reference.
This system can then build relationship models based on how often targets are seen interacting with each other and how they are socially interconnected. That is not just ‘keeping track’ of someone, but a full-on and constant surveillance, which would have cost a lot of human resources in the past. Now the manpower can be delegated to some button clicking.
There is a lot of resemblance to the patent we have talked about in our earlier post, where more detailed relationship models were seen as an important element in ‘smart policing’.
Here is an excerpt chart from a patent filed by the Chinese techcompany Hisense in 2019 on the importance and methodology of relationship models between targets of surveillance called “A holographic file structure of key personnel based on knowledge graph Construction methods and devices”.
The patent states: “The invention uses a knowledge graph to extract entities and relationships related to a target individual, applies multiple reasoning approaches to obtain multi-dimensional analysis, and builds a holographic profile to improve personnel management by addressing issues like mobility, poor timeliness, and high oversight rates.”
Bureaucratic paperwork
What is all this functionality worth if you can't compile reports to pass up the command chain!?
A ‘Smart report’ functionality provides the ability to produce MS Excel compatible data sheets. Via SQL queries from a locally hosted database, data can be accumulated and fill document templates for export.
Though the functionality hasn't been fully implemented here, evident by the many ‘formatting fragments’ in the output text, it still gives a good overview of what information the “operator” is most interested in.
Aside from the normal statistical standard indicators of “amount of residing foreigners of country X” or “what is their purpose of residency”, one graph stuck out. It showed a particular focus on citizens from the ‘Five Eyes Countries’. This seem to indicate that the security organizations in China have a blanket suspicion of every citizen from the UK, USA, Australia, New Zealand and Canada.
Conclusion of PART 1
The proliferation of Chinese digital surveillance inside the country over the past 15 years or so is not exactly a big surprise. Countless articles and broadcast TV pieces reiterated it.
During the Zero Covid-policy we witnessed how this system was pushed into overdrive (with mixed results). I experienced the effects myself working as a journalist in China traveling in Xinjiang, Inner Mongolia, Wuhan or other ‘sensitive places’.
The web-based dashboard in this article is most likely one manifestation of the famous Xueliang (雪亮 / Sharp Eyes) project that China has been working on for the past 10 years or so. Spawning out of SkyNet and other mass surveillance systems, it is aimed to be the ultimate, nation wide surveillance tool. It started out with the focus on video surveillance and gradually developed into a more holistic system, taking in all sorts of complementary datasets to build a complete profile of a target, enabling real time tracking and predictive policing. Although this demo seems to be mainly relying on video surveillance to drive its main functionality.
But the capabilities have steadily been growing. Gone are the days where the capabilities were limited to simply following a designated person via surveillance video from one corner of a specific town to the other.
Today, complex correlation models, self relying big-data analysis models, history based early warnings on individual behavior are the new thing. The dragnet is getting finer meshes as new data sources are increasingly added. The underlying systems and their interconnection are still experimented with, adjusted and improved.
While online surveillance and control has been equally growing in and outside of China, so does the real-world equivalent. In the logic of this underlying worldview humans are reduced to numbers, patterns and vector operations. A ‘Datamass’ that can be controlled, shaped and coerced as needed.
Keep on reading our PART 2 :
A few years ago a Swedish journalist lost her phone on the street and entered a police station to ask if it had been found. They told her to take a seat, and they turned on their computer screen and let her watch herself walking around town (Beijing) for the last several hours.
will you let people know if their profile is in the system?