Researchers at North Carolina State University announced on Mar. 24 that certain elements of artificial intelligence (AI) neural networks that contribute to data-privacy vulnerabilities are also essential for the models’ performance. The team used this discovery to develop a new technique aimed at better balancing model accuracy with privacy protection.
The research focuses on defending neural networks against membership inference attacks (MIAs), which are methods attackers use to determine if specific data was included in an AI model’s training set. “MIAs can jeopardize the privacy of individuals whose data was part of the training dataset,” said Xingli Fang, first author of the study and a Ph.D. student at North Carolina State University. “For example, if an attacker has partial data from an individual, it could use an MIA to determine if an AI model was trained using data from that individual.”
Jung-Eun Kim, corresponding author and assistant professor of computer science at NC State, said, “And if the individual’s data was used to train that model, the attacker could then infer the rest of the user’s information. Basically, MIAs pose a privacy vulnerability.” The researchers examined ‘weight parameters,’ which serve as connections within neural networks and play a key role in processing input data.
“When we started this project, we wanted to get a better understanding of which weight parameters in a model are most important for protecting privacy and which weight parameters are most important for performance,” Kim said. Fang added: “We found that only a few weight parameters represent a significant privacy vulnerability. However, we were surprised to learn that the vulnerable weight parameters are also among the most important weight parameters when it comes to performance. This means it is extremely difficult to reduce vulnerability risk without also hurting performance.” Despite this challenge, Fang continued: “However, we were able to use our new insights to develop a novel approach for improving data privacy by modifying the weight parameters and going through a fine-tuning process to adjust the model.”
The researchers tested their approach against four existing techniques using two advanced MIAs and found their method offered improved balance between maintaining privacy and preserving utility compared with previous solutions. Kim said: “We found that our approach achieves a better balance of privacy and performance relative to the previous techniques. We’re happy to talk with anyone in the field about how to incorporate this approach into their training.” Their paper will be presented at the Fourteenth International Conference on Learning Representations (ICLR2026) scheduled for April 23-27 in Rio de Janeiro.
