Thanks to Bots, Transparency Emerges as Lawmakers’ Choice for Regulating Algorithmic Harm

Digital conversational agents, like Amazon’s Alexa and Apple’s Siri, and communications agents, like those found on customer service website pages, seem to be everywhere.  The remarkable increase in the use of these and other artificial intelligence-powered “bots” in everyday customer-facing devices like smartphones, websites, desktop speakers, and toys, has been exceeded only by bots in the background that account for over half of the traffic visiting some websites.  Recently reported harms caused by certain bots have caught the attention of state and federal lawmakers.  This post briefly describes those bots and their uses and suggests reasons why new legislative efforts aimed at reducing harms caused by bad bots have so far been limited to arguably one of the least onerous tools in the lawmaker’s toolbox: transparency.

Bots Explained

Bots are software programmed to receive percepts from their environment, make decisions based on those percepts, and then take (preferably rational) action in their environment.  Social media bots, for example, may use machine learning algorithms to classify and “understand” incoming content, which is subsequently posted and amplified via a social media account.  Companies like Netflix uses bots on social media platforms like Facebook and Twitter to automatically communicate information about their products and services.

While not all bots use machine learning and other artificial intelligence (AI) technologies, many do, such as digital conversational agents, web crawlers, and website content scrappers, the latter being programmed to “understand” content on websites using semantic natural language processing and image classifiers.  Bots that use complex human behavioral data to identify and influence or manipulate people’s attitudes or behavior (such as clicking on advertisements) often use the latest AI tech.

One attribute many bots have in common is that their functionality resides in a black box.  As a result, it can be challenging (if not impossible) for an observer to explain why a bot made a particular decision or took a specific action.  While intuition can be used to infer what happens, secrets inside a black box often remain secret.

Depending on their uses and characteristics, bots are often categorized by type, such as “chatbot,” which generally describes an AI technology that engages with users by replicating natural language conversations, and “helper bot,” which is sometimes used when referring to a bot that performs useful or beneficial tasks.  The term “messenger bot” may refer to a bot that communicates information, while “cyborg” is sometimes used when referring to a person who uses bot technology.

Regardless of their name, complexity, or use of AI, one characteristic common to most bots is their use as agents to accomplish tasks for or on behalf of a real person.  This anonymity of agent bots makes them attractive tools for malicious purposes.

Lawmakers React to Bad Bots

While the spread of beneficial bots has been impressive, bots with questionable purposes have also proliferated, such as those behind disinformation campaigns used during the 2016 presidential election.  Disinformation bots, which operate social media accounts on behalf of a real person or organization, can post content to public-facing accounts.  Used extensively in marketing, these bots can receive content, either automatically or from a principal behind the scenes, related to such things as brands, campaigns, politicians, and trending topics.  When organizations create multiple accounts and use bots across those accounts to amplify each account’s content, the content can appear viral and attract attention, which may be problematic if the content is false, misleading, and biased.

The success of social media bots in spreading disinformation is evident in the degree to which they have proliferated.  Twitter recently produced data showing thousands of bot-run Twitter accounts (“Twitter bots”) were created before and during the 2016 US presidential campaign by foreign actors to amplify and spread disinformation about the campaign, candidates, and related hot-button campaign issues.  Users who received content from one of these bots would have had no apparent reason to know that it came from a foreign actor.

Thus, it’s easy to understand why lawmakers and stakeholders would want to target social media bots and those that use them.  In view of a recent Pew Research Center poll that found most Americans know about social media bots, and those that have heard about them overwhelmingly (80%) believe that such bots are used for malicious purposes, and with technologies to detect fake content at its source or the bias of a news source standing at only about 65-70 percent accuracy, politicians have plenty of cover to go after bots and their owners.

Why Use Transparency to Address Bot Harms?

The range of options for regulating disinformation bots to prevent or reduce harm could include any number of traditional legislative approaches.  These include imposing on individuals and organizations various specific criminal and civil liability standards related to the performance and uses of their technologies; establishing requirements for regular recordkeeping and reporting to authorities (which could lead to public summaries); setting thresholds for knowledge, awareness, or intent (or use of strict liability) applied to regulated activities; providing private rights of action to sue for harms caused by a regulated person’s actions, inactions, or omissions; imposing monetary remedies and incarceration for violations; and other often seen command-and-control style governance approaches.  Transparency, which is another tool lawmakers could deploy, could impose on certain regulated persons and entities that they provide information publicly or privately to an organization’s users or customers through a mechanism of notice, disclosure, and/or disclaimer (among other techniques).

Transparency is a long-used principal of democratic institutions that try to balance open and accountable government action and the notion of free enterprise with the public’s right to be informed.  Examples of transparency may be found in the form of information labels on consumer products and services under consumer laws, disclosure of product endorsement interests under FTC rules, notice and disclosures in financial and real estate transactions under various related laws, employee benefits disclosures under labor and tax laws, public review disclosures in connection with laws related to government decision-making, property ownership public records disclosures under various tax and land ownership/use laws, various healthcare disclosures under state and federal health care laws, and laws covering many other areas of public life.  Of particular relevance to the disinformation problem noted above, and why transparency seems well-suited to social media bots, is current federal campaign finance laws that require those behind political ads to reveal themselves.  See 52 USC §30120 (Federal Campaign Finance Law; publication and distribution of statements and solicitations; disclaimer requirements).

A recent example of a transparency rule affecting certain bot use cases is California’s bot law (SB-1001; signed by Gov. Brown on September 28, 2018).  The law, which goes into effect July 2019, will, with certain exceptions, make it unlawful for any person (including corporations or government agencies) to use a bot to communicate or interact with another person in California online with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election.  A person using a bot will not be liable, however, if the person discloses using clear, conspicuous, and reasonably designed notice to inform persons with whom the bot communicates or interacts that it is a bot.  Similar federal legislation may follow, especially if legislation proposed this summer by Sen. Diane Feinstein (D-CA) and legislative proposals by Sen. Warner and others gain traction in Congress.

So why would lawmakers choose transparency to regulate malicious bot technology use cases rather than use an approach that is arguably more onerous?  One possibility is that transparency is seen as minimally controversial, and therefore less likely to cause push-back by those with ties to special interests that might negatively respond to lawmakers who advocate for tougher measures.  Or, perhaps lawmakers are choosing a minimalist approach just to demonstrate that they are taking action (versus the optics associated with doing nothing).  Maybe transparency is seen as a shot across the bow warning to industry leaders: work hard to police themselves and those that use their platforms by finding technological solutions to preventing the harms caused by bots or else be subject to a harsher regulatory spotlight.  Whatever the reason(s), even something viewed as relatively easy to implement as transparency is not immune from controversy.

Transparency Concerns

The arguments against the use of transparency applied to bots include loss of privacy, unfairness, unnecessary disclosure, and constitutional concerns, among others.

Imposing transparency requirements can potentially infringe upon First Amendment protections if drafted with one-size-fits-all applicability.  Even before California’s bots measure was signed into law, for example, critics warned of the potential chilling effect on protected speech if anonymity is lifted in the case of social media bots.

Moreover, transparency may be seen as unfairly elevating the principals of openness and accountability over notions of secrecy and privacy.  Owners of agent-bots, for example, would prefer to not to give up anonymity when doing so could expose them to attacks by those with opposing viewpoints and cause more harm than the law prevents.

Both concerns could be addressed by imposing transparency in a narrow set of use cases and, as in California’s bot law, using “intent to mislead” and “knowingly deceiving” thresholds for tailoring the law to specific instances of certain bad behaviors.

Others might argue that transparency places too much of the burden on users to understand the information being disclosed to them and to take appropriate responsive actions.  Just ask someone who’s tried to read a financial transaction disclosure or a complex Federal Register rule-making analysis whether the transparency, openness and accountability actually made a substantive impact on their follow-up actions.  Similarly, it’s questionable whether a recipient of bot-generated content would investigate the ownership and propriety of every new posting before deciding whether to accept the content’s veracity, or whether a person engaging with an AI chatbot would forgo further engagement if he or she were informed of the artificial nature of the engagement.

Conclusion

The likelihood that federal transparency laws will be enacted to address the malicious use of social media bots seems low given the current political situation in the US.  And with California’s bots disclosure requirement not becoming effective until mid-2019, only time will tell whether it will succeed as a legislative tool in addressing existing bot harms or whether the delay will simply give malicious actors time to find alternative technologies to achieve their goals.

Even so, transparency appears to be a leading governance approach, at least in the area of algorithmic harm, and could become a go-to approach to governing harms caused by other AI and non-AI algorithmic technologies due to its relative simplicity and ability to be narrowly tailored.  Transparency might be a suitable approach to regulating certain actions by those who publish face images using generative adversarial networks (GANs), those who create and distribute so-called “deep fake” videos, and those who provide humanistic digital communications agents, all of which involve highly-realistic content and engagements in which a user could easily be fooled into believing the content/engagement involves a person and not an artificial intelligence.

AI’s Problems Attract More Congressional Attention

As contentious political issues continue to distract Congress before the November midterm elections, federal legislative proposals aimed at governing artificial intelligence (AI) have largely stalled in the Senate and House.  Since December 2017, nine AI-focused bills, such as the AI Reporting Act of 2018 (AIR Act) and the AI in Government Act of 2018, have been waiting for congressional committee attention.  Even so, there has been a noticeable uptick in the number of individual federal lawmakers looking at AI’s problems, a sign that the pendulum may be swinging in the direction favoring regulation of AI technologies.

Those lawmakers taking a serious look at AI recently include Mark Warner (D-VA) and Kamala Harris (D-CA) in the Senate, and Will Hurd (R-TX) and Robin Kelly (D-IL) in the House.  Along with others in Congress, they are meeting with AI experts, issuing new policy proposals, publishing reports, and pressing federal officials for information about how government agencies are addressing AI problems, especially in hot topic areas like AI model bias, privacy, and malicious uses of AI.

Sen. Warner, for example, the Senate Intelligence Committee Vice Chairman, is examining how AI technologies power disinformation.  In a draft white paper first obtained by Axios, Warner’s “Potential Policy Proposals for Regulation of Social Media and Technology Firms” raises concerns about machine learning and data collection, mentioning “deep fake” disinformation tools as one example.  Deep fakes are neural network models that can take images and video of people containing one type of content and superimpose them over different images and videos of other (or the same) people in a way that changes the original’s content and meaning.  To the viewer, the altered images and videos look like the real thing, and many who view them may be fooled into accepting the false content’s message as truth.

Warner’s “suite of options” for regulating AI include one that would require platforms to provide notice when users engage with AI-based digital conversational assistants (chatbots) or visit a website the publishes content provided by content-amplification algorithms like those used during the 2016 elections.  Another Warner proposal includes modifying the Communications Decency Act’s safe harbor provisions that currently protects social media platforms who publish offending third-party content, including the aforementioned deep fakes.  This proposal would allow private rights of action against platforms who fail to take steps, after notice from victims, that prevent offending content from reappearing on their sites.

Another proposal would require certain platforms to make their customer’s activity data (sufficiently anonymized) available to public interest researchers as a way to generate insight from the data that could “inform actions by regulators and Congress.”  An area of concern is the commercial use, by private tech companies, of their user’s behavior-based data (online habits) without using proper research controls.  The suggestion is that public interest researchers would evaluate a platform’s behavioral data in a way that is not driven by an underlying for-profit business model.

Warner’s privacy-centered proposals include granting the Federal Trade Commission with rulemaking authority, adopting GDPR-like regulations recently implemented across the European Union states, and setting mandatory standards for algorithmic transparency (auditability and fairness).

Repeating a theme in Warner’s white paper, Representatives Hurd and Kelly conclude that, even if AI technologies are immature, they have the potential to disrupt every sector of society in both anticipated and unanticipated ways.  In their “Rise of the Machines: Artificial Intelligence and its Growing Impact on U.S. Policy” report, the co-chairs of the House Oversight and Government Reform Committee make several observations and recommendations, including the need for political leadership from both Congress and the White House to achieve US global dominance in AI, the need for increased federal spending on AI research and development, means to address algorithmic accountability and transparency to remove bias in AI models, and examining whether existing regulations can address public safety and consumer risks from AI.  The challenges facing society, the lawmakers found, include the potential for job loss due to automation, privacy, model bias, and malicious use of AI technologies.

Separately, Representatives Adam Schiff (D-CA), Stephanie Murphy (D-FL), and Carlos Curbelo (R-FL), in a September 13, 2018, letter to the Director of National Intelligence, are requesting the Director of National Intelligence provide Congress with a report on the spread of deep fakes (aka “hyper-realistic digital forgeries”), which they contend are allowing “malicious actors” to create depictions of individuals doing or saying things they never did, without those individuals’ consent or knowledge.  They want the intelligence agency’s report to touch on everything from assessing how foreign governments could use the technology to harm US national interests, what sort of counter-measures could be deployed to detect and deter actors from disseminating deep fakes, and if the agency needs additional legal authority to combat the problem.

In a September 17, 2018, letter to the Equal Employment Opportunity Commission, Senators Harris, Patty Murray (D-WA), and Elizabeth Warren (D-MA) ask the EEOC Director to address the potentially discriminatory impacts of facial analysis technologies in the enforcement of workplace anti-discrimination laws.  As reported on this website and elsewhere, machine learning models behind facial recognition may perform poorly if they have been trained on data that is unrepresentative of data that the model sees in the wild.  For example, if training data for a facial recognition model contains primarily white male faces, the model may perform well when it sees new white male faces, but may perform poorly when it sees non-white male faces.  The Senators want to know if such technologies amplify bias in race, gender, disadvantaged, and vulnerable groups, and they have tasked the EEOC with developing guidelines for employers concerning fair uses of facial analysis technologies in the workplace.

Also on September 17, 2018, Senators Harris, Richard Blumenthal (D-CT), Cory Booker (D-NJ), and Ron Wyden (D-OR), sent a similar letter to the Federal Trade Commission, expressing concerns that the bias in facial analysis technologies could be considered unfair or deceptive practices under the Federal Trade Commission Act.  Stating that “we cannot wait any longer to have a serious conversation about how we can create sound policy to address these concerns,” the Senators urge the FTC to commit to developing a set of best practices for the lawful, fair, and transparent use of facial analysis.

Senators Harris and Booker, joined by Senator Cedric Richmond (D-LA), also sent a letter on September 17, 2018, to FBI Director Christopher Wray asking for the status of the FBI’s response to a 2016 General Accounting Office (GAO) comprehensive report detailing the FBI’s use of face recognition technology.

The increasing attention directed toward AI by individual federal lawmakers in 2018 may merely reflect the politics of the moment rather than signal a momentum shift toward substantive federal command and control-style regulations.  But as more states join those states that have begun enacting, in the absence of federal rules, their own laws addressing AI technology use cases, federal action may inevitably follow, especially if more reports of malicious uses of AI, like election disinformation, reach more receptive ears in Congress.

Trump Signs John S. McCain National Defense Authorization Act, Provides Funds for Artificial Intelligence Technologies

By signing into law the John S. McCain National Defense Authorization Act for Fiscal Year 2019 (H.R.5515; Public Law No: 115-232; Aug. 13, 2018), the Trump Administration has established a strategy for major new national defense and national security-related initiatives involving artificial intelligence (AI) technologies.  Some of the law’s $717 billion spending authorization for fiscal year 2019 includes proposed funding to assess the current state of AI and deploy AI across the Department of Defense (DOD).  The law also recognizes that fundamental AI research is still needed within the tech-heavy military services.  The law encourages coordination between DOD activities and private industry at a time when some Silicon Valley companies are being pressured by their employees to stop engaging with DOD and other government agencies in AI.

In Section 238 of the law, the Secretary of Defense is to lead “Joint Artificial Intelligence Research, Development, and Transition Activities” to include developing a set of activities within the DOD involving efforts to develop, mature, and transition AI technologies into operational use.  In Section 1051 of the law, an independent “National Security Commission on Artificial Intelligence” is to be established within the Executive Branch to review advances in AI and associated technologies, with a focus on machine learning (ML).

The Commission’s mandate is to review methods and means necessary to advance the development of AI and associated technologies by the US to comprehensively address US national security and defense needs.  The Commission is to review the competitiveness of the US in AI/ML and associated technologies.

“Artificial Intelligence” is defined broadly in Sec. 238 to include the following: (1) any artificial system that performs tasks under varying and unpredictable circumstances without significant human oversight, or that can learn from experience and improve performance when exposed to data sets; (2) an artificial system developed in computer software, physical hardware, or other context that solves tasks requiring human-like perception, cognition, planning, learning, communication, or physical action; (3) an artificial system designed to think or act like a human, including cognitive architectures and neural networks; (4) a set of techniques, including machine learning, that is designed to approximate a cognitive task; and (5) an artificial system designed to act rationally, including an intelligent software agent or embodied robot that achieves goals using perception, planning, reasoning, learning, communicating, decision making, and acting.  Section 1051 has a similar definition.

The law does not overlook the need for governance of AI development activities, and requires regular meetings of appropriate DOD officials to integrate the functional activities of organizations and elements with respect to AI; ensure there are efficient and effective AI capabilities throughout the DOD; and develop and continuously improve research, innovation, policy, joint processes, and procedures to facilitate the development, acquisition, integration, advancement, oversight, and sustainment of AI throughout the DOD.  The DOD is also tasked with studying AI to make recommendations for legislative action relating to the technology, including recommendations to more effectively fund and organize the DOD in areas of AI.

For further details, please see this earlier post.

Advanced Driver Monitoring Systems and the Law: Artificial Intelligence for the Road

Artificial intelligence technologies are expected to usher in a future where fully autonomous vehicles take people to their destinations without direct driver interaction.  During the transition from driver to driverless cars, roads will be filled with highly autonomous vehicles (HAVs) in which drivers behind the wheel are required to take control of vehicle operations at a moment’s notice. This is where AI-based advanced driver monitoring systems (DMS) play a role: ensuring HAV drivers are paying attention.  As big automakers incorporate advanced DMS into more passenger cars, policymakers will seek to ensure that these systems meet acceptable performance and safety standards as well as address issues such as privacy and cybersecurity related to use cases for the technology.  In this post, the technology behind advanced DMS is summarized followed by a brief summary of current governance efforts aimed at the technology.

The term “driver monitoring system,” also sometimes called “driver attention monitor” or “driver vigilance monitoring,” refers to a holistic system for analyzing driver behavior.  The principal goal of advanced DMS (as is the case for “older” DMS) is to return a warning or stimulation to alert and refocus the driver’s attention on the driving task.  In HAVs, advanced DMS is used to prepare the driver to re-take control of the vehicle under specified conditions or circumstances.

In operation, the technology detects behavior patterns indicative of the driver’s level of attention, fatigue, micro-sleep, cognitive load, and other physiological states. But the same technology can also be used for driving/driver experience personalization, such as customizing digital assistant interactions, music selection, route selection, and in-cabin environment settings.

Older DMS was adopted around 2006 with the introduction of electronic stability control, blind spot detection, forward collision warning, and lane departure warning technologies, among others, which indirectly monitor a driver by monitoring a driver’s vehicle performance relative to its environment.  Some of these systems were packaged with names like “drowsy driver monitoring,” “attention assist,” and others.

Advanced DMS technology began appearing in US commercial passenger vehicles starting in 2017.  Advanced DMS is expected to be used in SAE Levels 2 through Level 4 HAVs.  DMS in any form may not be needed for safety purposes once fully autonomous Level 5 is achieved, but the technology will likely continue to be used for personalization purposes even in Level 5 vehicles (which are reportedly not expected to be seen on US roadways until 2025 or later).

Advanced DMS generally tracks a driver’s head and hand positions, as well as the driver’s gaze (i.e., where the driver is looking), but it could also assess feet positions and posture relative to the driver’s seatback.  Cameras and touch sensors provide the necessary interface.  Advanced DMS may also utilize a driver’s voice using far-field speaker array technology and may assess emotion and mood (from facial expressions) and possibly other physiological states using various proximate and remote sensors.  Data from these sensors may be combined with signals from steering angle sensors, lane assist cameras, RADAR, LIDAR, and other sensor signals already available.

Once sensor signal data are collected, machine learning and deep neural networks may process the data.  Computer vision models (deep neural nets), for example, may be used for face/object detection within the vehicle.  Machine learning natural language processing models may be used to assess a driver’s spoken words.  Digital conversational assistant technology may be used to perform speech to text.  Knowledge bases may provide information to allow advanced DMS to take appropriate actions.  In short, much of the same AI tech used in existing human-machine interface (HMI) applications today can be employed inside passenger vehicles as part of advanced DMS.

From a regulatory perspective, in 2016, 20 states had introduced some sort of autonomous vehicle legislation.  In 2017, that number had jumped to 33 states.  No state laws, however, currently mandate the use of advanced DMS.

At the US federal government level, the US National Transportation Safety Board (NTSB), an independent agency that investigates transportation-related accidents, reported that overreliance on the semi-autonomous (Level 2) features of an all-electric vehicle and prolonged driver disengagement from the driving task contributed to a fatal crash in Florida in 2016.  In its report, the NTSB suggested the adoption of more effective monitoring of driver inattention commensurate with the capability level of the automated driving system.  Although the NTSB’s report does not rise to the level of a regulatory mandate for advanced DMS (the National Highway Transportation Safety Administration (NHTSA) sets transportation regulations), and applicable statutory language prohibits the admission into evidence or use of any part of an NTSB report related to an accident in a civil action for damages resulting from a matter mentioned in the report, the Board’s conclusions regarding probable cause and recommendations regarding preventing future accidents likely play a role in decisions by carmakers about deploying advanced DMS.

As for the NHTSA itself, while it has not yet promulgated advanced DMS regulations, it did publish an Automated Driving Systems, Vision 2.0: A Vision for Safety, report in September 2017.  While the document is clear that its intent is to provide only voluntary guidance, it calls for the incorporation of HMI systems for driver engagement monitoring, considerations of ways to communicate driving-related information as part of HMI, and encourages applying voluntary guidance from other “relevant organizations” to HAVs.

At the federal legislative level, H.R. 3388, the Safely Ensuring Lives Future Deployment and Research In Vehicle Evolution Act (SELF DRIVE Act) of 2017, contains provisions that would require the Department of Transportation (DOT) to produce a Safety Priority Plan that identifies elements of autonomous vehicles that may require standards.  More specifically, the bill would require NHTSA to identify elements that may require performance standards including HMI, sensors, and actuators, and consider process and procedure standards for software and cybersecurity as necessary.

In Europe, the European New Car Assessment Programme (Euro NCAP), Europe’s vehicle safety ratings and testing body, published its Roadmap 2025: Pursuit of Vision Zero, in September 2017.  In it, the safety testing organization addressed how its voluntary vehicle safety rating system is to be applied to HAVs in Europe.  In particular, the Euro NCAP identifies DMS as a “primary safety feature” standard beginning in 2020 and stated that the technology would need to be included in any new on-road vehicle if the manufacturer wanted to achieve a 5-star safety rating.  Manufacturers are already incorporating advanced DMS in passenger vehicles in response to the Euro NCAP’s position.

Aside from safety standards, advanced DMS may also be subject to federal and state statutory and common laws in the areas of product liability, contract, and privacy laws.  Privacy laws, in particular, will likely need to be considered by those employing advanced DMS in passenger vehicles due to the collection and use of driver and passenger biometric information by DMS.

Legislators, Stockholders, Civil Right Groups, and a CEO Seek Limits on AI Face Recognition Technology

Following the tragic killings of journalists and staff inside the Capital Gazette offices in Annapolis, Maryland, in late June, local police acknowledged that the alleged shooter’s identity was determined using a facial recognition technology widely deployed by Maryland law enforcement personnel.  According to DataWorks Plus, the company contracted to support the Maryland Image Repository System (MIRS) used by Anne Arundel County Police in its investigation, its technology uses face templates derived from facial landmark points extracted from image face data to digitally compare faces to a large database of known faces.  More recent technology, relying on artificial intelligence models, have led to even better and faster image and video analysis used by federal and state law enforcement for facial recognition purposes.  AI-based models can process images and video captured by personal smartphones, laptops, home or business surveillance cameras, drones, and government surveillance cameras, including body-worn cameras used by law enforcement personnel, making it much easier to remotely identify and track objects and people in near-real time.

Recently, facial recognition use cases have led to privacy and civil liberties groups to speak out about potential abuses, with a growing vocal backlash aimed at body-worn cameras and facial recognition technology used in law enforcement surveillance.  Much of the concern centers around the lack of transparency in the use of the technology, potential issues of bias, and the effectiveness of the technology itself.  This has spurred state legislators in several states to seek to impose oversight, transparency, accountability, and other limitations on the tech’s uses.  Some within the tech industry itself have even gone so far as to place self-imposed limits on uses of their software for face data collection and surveillance activities.

Maryland and California are examples of two states whose legislators have targeted law enforcement’s use of facial recognition in surveillance.  In California, state legislators took a recent step toward regulating the technology when SB-1186 was passed by its Senate on May 25, 2018.  In remarks accompanying the bill, legislators concluded that “decisions about whether to use ‘surveillance technology’ for data collection and how to use and store the information collected should not be made by the agencies that would operate the technology, but by the elected bodies that are directly accountable to the residents in their communities who should also have opportunities to review the decision of whether or not to use surveillance technologies.”

If enacted, the California law would require, beginning July 1, 2019, law enforcement to submit a proposed Surveillance Use Policy to an elected governing body, made available to the public, to obtain approval for the use of specific surveillance technologies and the information collected by those technologies.  “Surveillance technology” is defined in the bill to include any electronic device or system with the capacity to monitor and collect audio, visual, locational, thermal, or similar information on any individual or group. This includes, drones with cameras or monitoring capabilities, automated license plate recognition systems, closed-circuit cameras/televisions, International Mobile Subscriber Identity (IMSI) trackers, global positioning system (GPS) technology, software designed to monitor social media services or forecast criminal activity or criminality, radio frequency identification (RFID) technology, body-worn cameras, biometric identification hardware or software, and facial recognition hardware or software.

The bill would prohibit a law enforcement agency from selling, sharing, or transferring information gathered by surveillance technology, except to another law enforcement agency. The bill would provide that any person could bring an action for injunctive relief to prevent a violation of the law and, if successful, could recover reasonable attorney’s fees and costs.  The bill would also establish procedures to ensure that the collection, use, maintenance, sharing, and dissemination of information or data collected with surveillance technology is consistent with respect for individual privacy and civil liberties, and that any approved policy be publicly available on the approved agency’s Internet web site.

With the relatively slow pace of legislative action, at least compared to the speed at which face recognition technology is advancing, some within the tech community have taken matters into their own hands.  Brian Brakeen, for example, CEO of Miami-based facial recognition software company Kairos, recently decided that his company’s AI software will not be made available to any government, “be it America or another nation’s.”  In a TechCrunch opinion published June 24, 2018, Brakeen said, “Whether or not you believe government surveillance is okay using commercial facial recognition in law enforcement is irresponsible and dangerous” because it “opens the door for gross misconduct by the morally corrupt.”  His position is rooted in the knowledge of how advanced AI models like his are created: “[Facial recognition] software is only as smart as the information it’s fed; if that’s predominantly images of, for example, African Americans that are ‘suspect,’ it could quickly learn to simply classify the black man as a categorized threat.”

Kairos is not alone in calling for limits.  A coalition of organizations against facial recognition surveillance published a letter on May 22, 2018, to Amazon’s CEO, Jeff Bezos, in which the signatories demanded that “Amazon stop powering a government surveillance infrastructure that poses a grave threat to customers and communities across the country. Amazon should not be in the business of providing surveillance systems like Rekognition to the government.”  The organizations–civil liberties, academic, religious, and others–alleged that “Amazon Rekognition is primed for abuse in the hands of governments. This product poses a grave threat to communities,” they wrote, “including people of color and immigrants….”

Amazon’s Rekognition system, first announced in late 2016., is a cloud-based platform for performing image and video analysis without the user needing a background in machine learning, a type of AI.  Among its many uses today, Rekognition reportedly allows a user to conduct near real-time automated face recognition, analysis, and face comparisons (assessing the likelihood that faces in different images are the same person), using machine learning models.

A few weeks after the coalition letter dropped, another group, this one a collection of individual and organizational Amazon shareholders, issued a similar letter to Bezos.  In it, the shareholders alleged that “[w]hile Rekognition may be intended to enhance some law enforcement activities, we are deeply concerned it may ultimately violate civil and human rights.”  Several Microsoft employees took a similar stand against Microsoft’s role in its software used by government agencies.

As long as questions surrounding transparency, accountability, and fairness in the use of face recognition technology in law enforcement continue to be raised, tech companies, legislators, and stakeholders will likely continue to react in ways that address immediate concerns.  This may prove effective in the short-term, but no one today can say what AI-based facial detection and recognition technologies will look like in the future or to what extent the technology will be used by law enforcement personnel.

Senate-Passed Defense Authorization Bill Funds Artificial Intelligence Programs

The Senate-passed national defense appropriations bill (H.R.5515, as amended), to be known as the John S. McCain National Defense Authorization Act for Fiscal Year 2019, includes spending provisions for several artificial intelligence technology programs.

Passed by a vote of 85-10 on June 18, 2018, the bill would include appropriations for the Department of Defense “to coordinate the efforts of the Department to develop, mature, and transition artificial intelligence technologies into operational use.” A designated Coordinator will serve to oversee joint activities of the services in the development of a Strategic Plan for AI-related research and development.  The Coordinator will also facilitate the acceleration of development and fielding of AI technologies across the services.  Notably, the Coordinator is to develop appropriate ethical, legal, and other policies governing the development and use of AI-enabled systems in operational situations. Within one year of enactment, the Coordinator is to complete a study on the future of AI in the context of DOD missions, including recommendations for integrating “the strengths and reliability of artificial intelligence and machine learning with the inductive reasoning power of a human.”

In other provisions, the Director of the Defense Intelligence Agency (DIA; based in Ft. Meade, MD) is tasked with submitting a report to Congress within 90 days of enactment that directly compares the capabilities of the US in emerging technologies (including AI) and the capabilities of US adversaries in those technologies.

The bill would require the Under Secretary for R&D to pilot the use of machine-vision technologies to automate certain human weapons systems manufacturing tasks. Specifically, tests would be conducted to assess whether computer vision technology is effective and at a level of readiness to perform the function of determining the authenticity of microelectronic parts at the time of creation through final insertion into weapon systems.

The Senate version of the 2019 appropriations bill replaces an earlier House version (passed 351-66 on May 24, 2018).

At the Intersection of AI, Face Swapping, Deep Fakes, Right of Publicity, and Litigation

Websites like GitHub, Reddit and others offer developers and hobbyists dozens of repositories containing artificial intelligence deep learning models, instructions for their use, and forums for learning how to “face swap,” a technique used to automatically replace a face of a person in a video with that of a different person. Older versions of face swapping, primarily used on images, have been around for years in the form of entertaining apps that offered results with unremarkable quality (think cut and paste at its lowest, and photoshop editing at a higher level). With the latest AI models, however, including deep neural networks, a video with a face-swapped actor–so-called “deep fake” videos–may appear so seamless and uncanny as to fool even the closest of inspections, and the quality is apparently getting better.

With only subtle clues to suggest an actor in one of these videos is fake, the developers behind them have become the target of criticism, though much of the criticism has also been leveled generally at the AI tech industry, for creating new AI tools with few restrictions on potential uses beyond their original intent.  These concerns have now reached the halls of New York’s state legislative body.

New York lawmakers are responding to the deep fake controversy, albeit in a narrow way, by proposing to make it illegal to use “digital replicas” of individuals without permission, a move that would indirectly regulate AI deep learning models. New York Assembly Bill No. A08155 (introduced in 2017, amended Jun. 5, 2018) is aimed at modernizing New York’s right of publicity law (N.Y. Civ. Rights Law §§ 50 and 50-1)–one of the nation’s oldest publicity rights laws that does not provide post-mortem publicity rights–though it may do little to curb the broader proliferation of face swapped and deep fake videos. In fact, only a relatively small slice of primarily famous New York actors, artists, athletes, and their heirs and estates would benefit from the proposed law’s digital replicas provision.

If enacted, New York’s right of publicity law would be amended to address computer-generated or electronic reproductions of a living or deceased individual’s likeness or voice that “realistically depicts” the likeness or voice of the individual being portrayed (“realistic” is undefined). Use of a digital replica would be a violation of the law if done without the consent of the individual, if the use is in a scripted audiovisual or audio work (e.g., movie or sound recording), or in a live performance of a dramatic work, that is intended to and creates the clear impression that the individual represented by the digital replica is performing the activity for which he or she is known, in the role of a fictional character.

It would also be a violation of the law to use a digital replica of a person in a performance of a musical work that is intended to and creates the clear impression that the individual represented by the digital replica is performing the activity for which he or she is known, in such musical work.

Moreover, it would be a violation to use a digital replica of a person in an audiovisual work that is intended to and creates the clear impression that an athlete represented by the digital replica is engaging in an athletic activity for which he or she is known.

The bill would exclude, based on First Amendment principles, a person’s right to control their persona in cases of parody, satire, commentary, and criticism; political, public interest, or newsworthy situations, including a documentary, regardless of the degree of fictionalization in the work; or in the case of de minimis or incidental uses.

In the case of deep fake digital replicas, the bill would make it a violation to use a digital replica in a pornographic work if done without the consent of the individual if the use is in an audiovisual pornographic work in a manner that is intended to and creates the impression that the individual represented by the digital replica is performing.

Similar to the safe harbor provisions in other statutes, the New York law would provide limited immunity to any medium used for advertising including, but not limited to, newspapers, magazines, radio and television networks and stations, cable television systems, billboards, and transit advertising, that make unauthorized use of an individual’s persona for the purpose of advertising or trade, unless it is established that the owner or employee had knowledge of the unauthorized use, through presence or inclusion, of the individual’s persona in such advertisement or publication.

Moreover, the law would provide a private right of action for an injured party to sue for an injunction and to seek damages. Statutory damages in the amount of $750 would be available, or compensatory damages, which could be significantly higher.  The finder of fact (judge or jury) could also award significant “exemplary damages,” which could be substantial, to send a message to others not to violate the law.

So far, AI tech developers have largely avoided direct legislative or regulatory action targeting their AI technologies, in part because some have taken steps to self-regulate, which may be necessary to avoid the confines of command and control-style state or federal regulatory schemes that would impose standards, restrictions, requirements, and the right to sue to collect damages and collect attorneys’ fees. Tech companies efforts at self-regulating, however, have been limited to expressing carefully-crafted AI policies for themselves and their employees, as well as taking a public stance on issues of bias, ethics, and civil rights impacts from AI machine learning. Despite those efforts, more laws like New York’s may be introduced at the state level if AI technologies are used in ways that have questionable utility or social benefits.

For more about the intersection of right of publicity laws and regulating AI technology, please see an earlier post on this website, available here.

Obama, Trump, and the Regulation of Artificial Intelligence

Near the end of his second term, President Obama announced a series of workshops and government working groups tasked with “Preparing for the Future of Artificial Intelligence.” Then, just weeks before the 2016 presidential general election, the Obama administration published two reports including one titled “The National Artificial Intelligence Research and Development Plan.” In it, Obama laid out seven strategies for AI-related R&D, including making long-term investments in AI research to enable the United States to remain a world leader in AI, developing effective methods for human-AI interaction, and ensuring the safety, security, and trustworthiness of AI systems. The Obama AI plan also included strategies for developing shared and high-quality public datasets and environments for AI training and testing, creating standards and benchmarks for evaluating AI technologies, and understanding the national AI research workforce needs. His plan also recognized the need for collaboration among researchers to address the ethical, legal, and societal implications of AI, topics that still resonate today.

Two years after Obama’s AI announcement, the Trump administration in May 2018 convened an Artificial Intelligence Summit at the White House and then published an “Artificial Intelligence for the American People” fact sheet highlighting President Trump’s AI priorities. The fact sheet highlights the President’s goal of funding fundamental AI R&D, including in the areas of computing infrastructure, machine learning, and autonomous systems. Trump’s AI priorities also include a focus on developing workforce training in AI, seeking a strategic military advantage in AI, and leveraging AI technology to improve efficiency in delivering government services. The Trump fact sheet makes no mention of Obama’s AI plan.

Despite some general overlap and commonality between Obama’s and Trump’s AI goals and strategies, such as funding for AI, workforce training, and maintaining the United States’ global leadership in AI, one difference stands out in stark contrast: regulating AI technology. While Obama’s AI strategy did not expressly call for regulating AI, it nonetheless recognized a need for setting regulatory policy for AI-enabled products. To that end, Obama recommended drawing on appropriate technical expertise at the senior level of government and recruiting the necessary AI technical talent as necessary to ensure that there are sufficient technical seats at the table in regulatory policy discussions.

Trump, on the other hand, has rolled back regulations across the board in a number of different governmental areas and, in the case of AI, has stated that he would seek to “remove regulatory barriers” to AI innovation to foster new American industries and deployment of AI-powered technologies. With the Trump administration’s express concerns about China’s plan to dominate high tech, including AI, by 2025, as well as Congressional efforts at targeted AI legislation slowed in various committees, any substantive federal action toward regulating AI appears to be a long way off. That should be good news to many in the US tech industry who have long resisted efforts to regulate AI technologies and the AI industry.

Congress Looking at Data Science for Ways to Improve Patent Operations

When Congress passed the sweeping Leahy-Smith America Invents Act (AIA) on September 16, 2011, legislators weren’t concerned about how data analytics might improve efficiencies at one of the Commerce Department’s most data-heavy institutions: the US Patent Office. Patent reformers at the time were instead focused on curtailing patent troll litigation and conforming aspects of US patent law to those of other countries. Consequently, the Patent Office’s trove of pre-classified, pre-labeled, and semi-structured patent application and invention data–information ripe for big data analytics–remained mostly untapped at the time.

Fast forward to 2018 and Congress has finally put patent data in its cross-hairs. Now, Congress wants to see how “advanced data science analytics” techniques, such as artificial intelligence, machine learning, and other methods, could be used to analyze patent data and make policy recommendations. If enacted, the “Building Innovation Growth through Data for Intellectual Property Act” or the “BIG Data for IP Act” of 2018 (S. 2601; sponsored by Sen. Coons and Sen. Hatch) would require an investigation into how data science could help the Patent Office understand its current capabilities and whether its information technology systems need modernizing.

Those objectives, however, may be too narrow.  Silicon Valley tech companies, legal tech entrepreneurs, and even students have already seized upon the opportunities big patent data and machine learning techniques present, and, as a result, have developed interesting and useful capabilities.

Take, for example, the group of Stanford University students who in late 2011 developed a machine learning technique to automatically classify US patent applications based on an application’s written invention description. The students, part of Stanford’s CS229 Machine Learning class, proposed their solution around the same time Senators Leahy, Smith, and the rest of Congress were debating the AIA in the fall of 2011.  More recently, AI technologies used by companies like Cloem, AllPriorArt, AllPriorClaims, RoboReview, Specif.io, and others have shown how patent data and AI can augment traditional patent practitioner’s roles in the legal services industry.

Some of these AI tools may one day reduce much of the work patent practitioners have traditionally performed and could lead to fewer Examiners at the Patent Office whose jobs are to review patent applications for patentability. Indeed, some have imagined a world in which advanced machine learning models conceive inventions and prepare and file a patent application to protect those ideas without further human input.  In the future, advanced machine learning models, trained on the “prior art” patent data, could routinely examine patent applications for patentability, thus eliminating the need for costly and time-consuming inter partes reviews (a trial-like proceeding that has created much uncertainty since enactment of the AIA).

So perhaps Congress’ BIG Data for IP Act should focus less on how advanced data analytics can be used to “improve consistency, detect common sources of error, and improve productivity,” as the bill is currently written, and focus more globally on how patent data, powering new AI models, will disrupt Patent Office operations, the very nature of innovation, and how patent applications are prepared, filed, and examined.

In Your Face Artificial Intelligence: Regulating the Collection and Use of Face Data (Part II)

The technologies behind “face data” collection, detection, recognition, and affect (emotion) analysis were previously summarized. Use cases for face data, and reported concerns about the proliferation of face data collection efforts and instances of face data misuse were also briefly discussed.

In this follow-on post, a proposed “face data” definition is explored from a governance perspective, with the purpose of providing more certainty as to when heightened requirements ought to be imposed on those involved in face data collection, storage, and use.  This proposal is motivated in part by the increased risk of identity theft and other instances of misuse from unauthorized disclosure of face data, but also recognizes that overregulation could subject persons and entities to onerous requirements.

Illinois’ decade-old Biometric Information Privacy Act (“BIPA”) (740 ILCS 14/1 (2008)), which has been widely cited by privacy hawks and asserted against social media and other companies in US federal and various state courts (primarily Illinois and California), provides a starting point for a uniform face data definition. The BIPA defines “biometric identifier” to include a scan of a person’s face geometry. The scope and meaning of the definition, however, remains ambiguous despite close scrutiny by several courts. In Monroy v. Shutterfly, Inc., for example, a federal district court found that mere possession of a digital photograph of a person and “extraction” of information from such photograph is excluded from the BIPA:

“It is clear that the data extracted from [a] photograph cannot constitute “biometric information” within the meaning of the statute: photographs are expressly excluded from the [BIPA’s] definition of “biometric identifier,” and the definition of “biometric information” expressly excludes “information derived from items or procedures excluded under the definition of biometric identifiers.”

Slip. op. No. 16-cv-10984 (N.D. Ill. 2017). Despite that finding, the Monroy court concluded that a “scan of face geometry” under the statute’s definition includes a “scan” of a person’s face from a photograph (or a live scan of a person’s face geometry). Although not at issue in Monroy, the court did not address whether that BIPA applies when a scan of any part of a person’s face geometry from an image is insufficient to identify the person in the image. That is, the Monroy holding arguably applies to any data made by a scan, even if that data by itself cannot lead to identifying anyone.

By way of comparison, the European Union’s General Data Protection Regulation (GDPR), which governs “personal data” (i.e., any information relating to an identified or identifiable natural person), will regulate biometric information when it goes into effect in late May 2018. Like the BIPA, the GDPR will place restrictions on “personal data resulting from specific technical processing relating to the physical, physiological or behavioural characteristics of a natural person, which allow or confirm the unique identification of that natural person, such as facial images or dactyloscopic data” (GDPR, Article 4) (emphasis added).  Depending on how EU nation courts interpret the GDPR generally, and Article 4 specifically, a process that creates any biometric data that relates to, or could lead to, or that allows one to identify a person, or allows one to confirm an identity of a person, is a potentially covered process under the GDPR.

Thus, to enhance clarity for potentially regulated individuals and companies dealing with US citizens, “face data” could be defined, as set forth below, in a way that considers a minimum quantity or quality of data below which a regulated entity would not be within the scope of the definition (and thus not subject to regulation):

“Face data” means data in the possession or control of a regulated entity obtained from a scan of a person’s face geometry or face attribute, as well as any information and data derived from or based on the geometry or attribute data, if in the aggregate the data in the possession or control of the regulated entity is sufficient for determining an identity of the person or the person’s emotional (physiological) state.

The term “determining an identity of the person or the person’s emotional (physiological) state” relates to any known computational or manual technique for identifying a person or that person’s emotions.

The term “is sufficient” is interpretable; it would need to be defined explicitly (or, as is often the case in legislation, left for the courts to fully interpret). The intent of “sufficient” is to permit the anonymization or deletion of data following the processing of video signals or images of a person’s face to avoid being categorized as possessing regulated face data (to the extent probabilistic models and other techniques could not be used to later de-anonymize or reconstruct the missing data and identify a person or that person’s emotional state). The burden of establishing the quality and quantity of face data that is insufficient for identification purposes should rest with the regulated entity that possesses or controls face data.

Face data could include data from the face of a “live” person captured by a camera (e.g., surveillance) as well as data extracted from existing media (e.g., stored images). It is not necessary, however, for the definition to encompass the mere virtual depiction or display of a person in a live video or existing image or video. Thus, digital pictures of friends or family on a personal smartphone would not be face data, and the owner of the phone should not be a regulated entity subject to face data governance. An app on that smartphone, however, that uses face detection algorithms to process the pictures for facial recognition and sends that data to a remote app server for storage and use (e.g., for extraction of emotion information) would create face data.

By way of other examples, a process involving pixel-level data extracted from an image (a type of “scan”) by a regulated entity  would create face data if that data, combined with any other data possessed or controlled by the entity, could be used in the aggregate to identify the person in the image or that person’s emotional state. Similarly, data and information reflecting changes in facial expressions by pixel-level comparisons of time-slice images from a video (also a type of scan) would be information derived from face data and thus would be regulated face data, assuming the derived data combined with other data owned or possessed could be used to identify the person in the image or the person’s emotional state.

Information about the relative positions of facial points based on facial action units could also be data derived from or based on the original scan and thus would be face data, assuming again that the data, combined with any other data possessed by a regulated entity, could be used to identify a person or that person’s emotional state. Classifications of a person’s emotional state (e.g., joy, surprise) based on extracted image data would also be information derived from or based on a person’s face data and thus would also be face data.

Features extracted using deep learning convolutions of an image of a person’s face could also be face data if the convolution information along with other data in the possession or control of a regulated entity could be used to identify a person or that person’s emotional state.

For banks and other institutions that use face recognition for authentication purposes, sufficient face data would obviously need to be in the banks possession at some point in time to positively identify a customer making a transaction. This could subject the institution to face data governance during that time period. In contrast, a social media platform that permits users to upload images of people but does not scan or otherwise process the images (such as by cross-referencing other existing data) would not create face data and thus would not subject the platform to face data governance, even if it also possessed tagged images of the same individuals in the uploaded images. Thus, the mere possession or control over images, even if the images could potentially contain identifying information, would not constitute face data. But, if a platform were to scan (process) the uploaded images for identification purposes or sell or provide the images uploaded by users to a third party that scans the images to extract face geometry or attributes data for purposes such as targeted advertising, could subject the platform and the third party to face data governance.

The proposed face data definition, which could be modified to include “body data” and “voice data,” is merely one example that US policymakers and stakeholders might consider in the course of assessing the scope of face data governance in the US.  The definition does not exclude the possibility that any number of exceptions, exclusions, and limitations could be implemented to avoid reaching actors and actions that should not be covered, while also maintaining consistency with existing laws and regulations. Also, the proposed definition is not intended to directly encompass specific artificial intelligence technologies used or created by a regulated entity to collect and use face data, including the underlying algorithms, models, networks, settings, hyper-parameters, processors, source code, etc.

In a follow-on post, possible civil penalties for harms caused by face data collection, storage, and use will be briefly considered, along with possible defenses a regulated person or entity may raise in litigation.