It's not just Sachin or Nora, why deepfakes are about to hit you and me

The issue of digital doppelgangers is looming large on everyone’s timeline since the deepfake video of Rashmika Mandanna went viral in November last year. The latest in this spate of celebrity deepfakes is Nora Fatehi. Last week, the Canadian dancer and actress based in India shared on Instagram that technical apparel and athletic shoes website Lululemon has used a deepfake video of her for promoting the brand. The other notable targets include Sachin Tendulkar, who in a video is seen promoting a mobile app, as well as Alia Bhatt, Kajol, Katrina Kaif, Priyanka Chopra, Ratan Tata and Prime Minister Narendra Modi.Watch: Celebrities' deepfakes circulating online medium nocaption107030379As cases mount, security agencies too are coming into the grips of handling such cases. On January 20, the Delhi Police arrested Eemani Naveen — a 24-year-old engineer from Andhra Pradesh’s Guntur district — the main accused behind the deepfake of Mandanna. Reportedly, Naveen told the police that he did so to boost the followers of Rashmika Mandanna's fan page, which he created. Before we look at why this trend is increasingly worrying lawmakers and the legislation that is coming up to curb its menace, it is important to look at what deepfakes are, and the dangers it poses from ruining people’s reputations to causing geopolitical stirs. There’s also no denying that the frequency of such cases are going to rise in times to come as technology gets more sophisticated.Sachin Tendulkar's post warning people about his deepfake: These videos are fake. It is disturbing to see rampant misuse of technology. Request everyone to report videos, ads & apps like these in large numbers.

Social Media platforms need to be alert and responsive to complaints. Swift action from their end is crucial to stopping the… pic.twitter.com/4MwXthxSOM

— Sachin Tendulkar (@sachin_rt) January 15, 2024 What are deepfakes?The concept of visual deception in media is not new. Earlier confined to the hallows of cinema, the advent of smartphones have made tools of visual deception widely available to the average consumer. Instead, we must ask, how are deepfakes different from typical photoshopping that has become an integral feature of social media?Deepfake is literally photoshopping on steroids. It uses machine power to process large amounts of existing data with the idea of creating hyper-realistic audios, videos or images that approximate the existing media without being an exact replica. Deepfake was revolutionised in 2014, with the refinement of the Generative Adversarial Network (GAN). • In GAN, two machine learning networks are utilised to develop synthetic content through an adversarial process. • The first network is the 'generator' that attempts to create new examples of that data which exhibit the same characteristics of the original data. • These generated examples are then presented to the second machine learning network, which attempts to detect flaws in the presented examples and rejects those which it determines do not exhibit the same sort of characteristics as the original data — identifying them as 'fakes'. • These fakes are then ‘returned’ to the first network, so it can learn to improve its process of creating new data. • This back and forth continues until the generator produces fake content that the adversary identifies as real. Why use of deepfakes is spikingReports predict the increase of deepfake in coming years and how it will try to outdo the other, with each pegging the rate of proliferation higher than the other. Estimates vary from a 230% increase to a 3000% increase. And why is this? Simply, because the technology used to generate sophisticated deepfake is increasingly becoming available. According to a study, it takes 25 minutes, and costs nothing, to create a minute long deepfake pornographic video using just one clear face image. medium nocaption107112626The proliferation of deepfakes across sectors, however, is also not uniform. 98% of the deepfake content relates to deepfake pornography. As such, women, especially those in the entertainment industry, seem to be the most targeted group of this. medium nocaption107112831What are the dangers of deepfakes?A number of risks arise from the trend of deepfakes, the foremost that has caught attention is disinformation. The spread of fake media corrodes trust in the fairness and impartiality of public processes. In extreme cases, it can lead to political discord and incite violence. Also, the presence of disinformation casts doubt on the accuracy of all information. This allows people to deflect genuine accusations also using the claim that it is false. Moreover, it will take more effort for people to discern what is genuine or not, thereby making them wary of sharing information and reducing the free flow of information.Apart from disinformation, deepfakes also have damaging effects on the reputation of the person featuring in it. Recently, the Delhi high court issued an injunction protecting Anil Kapoor’s personality rights and restraining others from using artificial intelligence to generate and use his image or likeness without consent. Also, because general deepfakes use personal information without the consent of the person, it violates their right to privacy. Use of deepfake in objectionable contexts also leads to cyberbullying and harassment. Despite the numerous harms, the response of the industry to this is struggling for a number of reasons. First, media uploaded online has a unique hash value which helps in identifying social media intermediaries in identifying these media. Any alteration to the media, however insignificant, ends up changing the hash value which leads to the offending media escaping takedown. Also, currently the most sophisticated AI detection models claim to identify almost 90% of deepfake content. However, given the sheer magnitude of deepfake content and the questionable rate of accuracy, this still means that a large amount of deepfake content is still online and genuine content may also be getting removed. Worryingly, deepfake generation process is getting better at a faster rate than deepfake detection methods. What has been the government’s response to this?Legal frameworks around the world are similarly struggling to develop an appropriate regulatory response to the issue. Most jurisdictions are trying to tackle deepfake harms under privacy laws along with the laws that provide remedial action for the particular harm that has been caused. Only a few jurisdictions such as China, European Union and some states in the US have laws that specifically deal with deepfakes. In India, legal action against deepfakes can be taken under the Information Technology Act, 2000; The Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021 and the Indian Penal Code, 1860. medium nocaption107112707Following the deepfake video of Rashmika Mandanna, there seemed to be contradictory stands being taken on whether a regulation was required or not to deal with deepfakes.As of now, the government following rounds of discussion with social media platforms has not come up with a specific directive against deepfakes, but has issued an advisory against the same:• Reiterates obligations on intermediaries.• Inform users about content that is not permitted at the time of first-registration and also as regular reminders, in particular, at every instance of login and while uploading/sharing information onto the platform.• In terms of service and user agreements, to highlight that intermediary has to report legal violations. • Is obliged to ensure reasonable efforts to prevent users from hosting, displaying, uploading, modifying, publishing, transmitting, storing, updating, or sharing deepfakes. The first video which is original and the second video which is #deepfake .

A deepfake video was going viral on SM in which #RashmikaMandanna 's face was edited.

AI is dangerous! Be safe girls! . . . . . . #RashmikaMandannahot #DelhiNCR #earthquakes #viralvideo … pic.twitter.com/HNvqYMDsTe

— Priya Sarkar (@QueenPinkSoul69) November 6, 2023 However, there are a number of concerns with this approach. First, it places greater emphasis on the intermediaries to proactively remove content that may be offending. This demonstrably leads to overcompliance and can have chilling effects on free speech. Moreover, it is compounded by the fact that the obligation is sought to be enforced by threatening that the platform will lose safe harbour immunity. Also, this approach only reacts to the immediate harm that is being witnessed, while ignoring other emergent harms. Regulating only social media intermediaries fails to consider that deepfakes have proliferated mediums other than social media, and pose other emergent threats. How it’s linked to fraud cases One of the reasons for this is that the slew of incidents that have been reported in the past couple of months have involved celebrities. This is because footage required for generating synthetic media of celebrities is easily available. But with the improvement in technology, lesser data will be required to generate this synthetic media. In that context, an eminent harm is that of financial frauds being committed using deepfakes. The World Economic Forum believes that almost two out of three cybersecurity professionals have experienced some form of deepfake attack within their respective organisation. At present, the UK is one of the worst hit countries by fraud due to impersonation. This is in part because English is the second most spoken language in the world, which allows scamsters worldwide to target the UK. Moreover, with advancements in AI, it will become easier to generate deepfakes that mimic people’s voices and accents in multiple languages. In India too, financial sector experts are anticipating deepfake fraud in the top five anticipated in the financial sector. This, especially so, given the already meteoric rise of online financial frauds from Rs 2,296 crore in 2022 to Rs 5,574 crore in 2023. One way in which this will percolate at the level of the average consumer is by way of identity verification. Post-pandemic, the financial sector resorted increasingly to eKYC, doing away with the need of presenting physical documents. This typically involves either submitting one’s selfie along with a picture of identity documents or a video call with an authorised personnel to verify the identity of a person. Video KYC is considered to be less prone to risk because of the “active liveness” component that checks whether the person in front of the camera is the actual user. However, there are growing concerns that remote biometric verification can be now easily exploited by deepfakes. Financial institutions rely on human judgement to assess the genuineness of the person undertaking the KYC. Also, with AI advancement we are at a stage where humans can no longer spot the difference between real faces and deepfakes. As per a study, participants were able to distinguish deepfakes only 48% of the time. What is more, people have a 7.7% higher tendency to trust deepfake images than real faces, putting in perspective the vulnerability of the current architecture of eKYC in India.Because eKYC has significantly decreased the cost of onboarding customers, by almost 90%, deepfakes can disrupt the current landscape of financial services in India. Other ways of defrauding include the use of clone apps to create hyper real videos, videos leading more authenticity than screenshots, of financial documents such as profit and loss statement, bank statements, ledgers etc to convince individuals that the fraudster is an authorized representative of the financial entity. Recently, Zerodha alerted its users against such a scam wherein scamsters were sharing bank statements to convince them into paying a 1.8 lakh registration fee for availing substantial returns. Synthetic identities can also be used to create new accounts that can be used for money laundering.medium nocaption107112874How we can tackle the issue As far as the financial sector goes, measures like multi-factor authentication, behavioural biometrics and zero trust reinforcement can mitigate frauds, but there is still a need to regulate the tech at the level of vendors for deepfake technology also. For example, China requires conspicuous labels for synthetically generated media. Similarly, the AI Act of the European Union has transparency requirements when artificial intelligence is used in certain contexts. In India, it is expected that the proposed Digital India Act will lay out enforceable standards for responsible AI in India. In doing so, there is a need to undertake wider public consultation on the emergent harms from generative AI and regulate them accordingly. Reactive policy framing often leads to patchwork and disjointed regulatory frameworks. In the case of deepfakes in India, this is evidenced by how even as MeitY has emphasised on deepfake regulation pertaining to some harms, financial regulators have not deliberated on it for harms arising within their specific sectors. Even as sectoral harms are better understood by the respective regulators, there is a need that MeitY lays down baseline frameworks for regulation of deepfake, if only for building a more cohesive framework. For this it needs to look beyond harms that make headlines.The writer is a research fellow at Vidhi Centre for Legal Policy