Written by Nathan Summers and Dr Sergio Coronado

On Wednesday, March 13th, 2024, the European Parliament voted to approve the AI Act after having reached a provisional consensus last December. This represents a significant milestone in the global regulation of Artificial Intelligence, as the EU AI Act is the first of its kind, anywhere in the world. Once the regulations are fully implemented, AI systems developed and deployed within Europe will be divided into four categories based on an assessment of their risk and will be subject to oversight on this basis (For more information on these risk categorizations and the corresponding legal obligations, please see our previous publication on the EU AI Act). The risk categories are as follows:

1. Unacceptable Risk: These are systems deemed too dangerous to ever be permitted within the EU. These include systems for social scoring and real-time public identification.
2. High Risk: These systems will need to be in strict compliance with certain obligations before being put to market. AI systems used in healthcare, vocational applications, or other critical infrastructure will be classified here.
3. Limited Risk: These systems will require deployers to be transparent about AI usage. This will apply to chatbots, AI-generated text and images, etc.
4. Minimal or No Risk: These systems require no further compliance. This category will encompass AI used in video games or spam filters, among others.

While it is undeniable that the AI Act will provide necessary protections for consumers, there is reasonable concern as to how it will impact the future of AI investment and development within Europe. Already, European investment into AI is significantly lower than in the United States (7% of US investment) or China (20% of Chinese investment) and there is fear that these regulations could create additional hurdles that stifle innovation, hinder growth, and deter investment into European AI firms.

When it comes to investment in AI, the United States leads the way, with over $29 billion in private funding dedicated to the sector last year. China follows closely behind with investments of almost $10 billion. However, the European Union (EU) is significantly behind, with investments amounting to just over $2 billion.

Although it remains to be seen whether this will be the case, some of Europe’s largest AI firms are already eyeing the US for expansion and investment opportunities.

Mistral AI, one of Europe’s most influential AI companies, has recently entered into a partnership with Microsoft with an apparent goal of expanding their operations into the US market. As one of Europe’s few AI “unicorns,” where Mistral leads, others are inclined to follow. Hungary’s OnCompass and Denmark’s Corti, both AI healthcare startups, have also recently expanded into the US. As both companies are set to be considered “High Risk” within the EU — and given the level of AI investment in the US — these moves are unsurprising.

In order to maintain competitiveness in the ever-growing AI market, it is crucial for the EU to offer compelling incentives and funding for firms to stay within the region. The European Commission’s initiative to provide European AI firms with access to EU supercomputers is a strong start, but alone will likely not go far enough. The question then remains: how best to ensure a competitive future for the EU AI market?

Expand Public Funding

Although Europe may lack the extensive venture capital infrastructure found in the US, it compensates with robust public funding initiatives. Horizon Europe stands out as a prime example, ensuring that European technology and innovation maintain their global competitiveness. Currently, public funding for AI research falls within this initiative, laying a strong foundation for advancement. However, this also implies that AI research will compete with other innovative endeavors for a share of the budget, presenting a potential challenge amidst competing priorities.

To address this challenge and to further incentivize AI firms to remain and develop in Europe, the EU should establish dedicated initiatives for the public funding of AI research and development. This will ensure that AI projects do not need to compete for funding within the Horizon Europe framework. By creating such a dedicated funding mechanism, the EU can provide tangible support to firms, promoting competitiveness and fostering innovation in the AI sector. This initiative will help make Europe an attractive destination for future AI research and development.

Foster Stakeholder Collaboration

Because of the far-reaching potential impact of AI, its stakeholders are uniquely interdisciplinary. Academics, entrepreneurs, regulators, industry, and the public are all invested in the success, but also the development, of AI technologies. These stakeholders hold a variety of positions on what AI should be used for, how it is regulated, and how its benefits and risks should be balanced. Furthermore, the expertise required to develop safe, robust, and effective AI goes beyond that of only computer scientists. To develop effective healthcare AI, healthcare professionals must be involved.

The EU can bolster its AI market by investing in initiatives that foster interdisciplinary collaboration among relevant domain experts and AI developers. This will ensure that AI development benefits from a wide range of perspectives and expertise. By organizing workshops, conferences, competitions, hackathons, and other events that encourage interdisciplinary dialogue and establishing collaborative platforms for knowledge sharing, the EU will foster the creation of AI technologies that are not only technologically advanced but also ethically robust, socially responsible, and aligned with societal needs. This emphasis on interdisciplinary collaboration enhances the EU’s position in the global AI market by cultivating innovation, fostering public trust, and driving sustainable growth.

Facilitating Data and Resource Availability

Key to the development of safe, robust, and effective AI is the availability of data. In line with the European Commission’s initiative to provide access to EU supercomputers, access to high quality data could provide a significant incentive to AI firms. This could be particularly true for startups and small companies, which may be unable to acquire such data themselves.

By generating, curating, and providing access to data, the EU will not only save AI firms the expense of sourcing this data themselves but will also save them the legal burden of ensuring GDPR compliance, as it can be safely assumed that any data the EU provides will be in compliance with its own laws. Such an initiative would thus minimize regulatory burdens while simultaneously stimulating the AI startup sector within Europe.

Strengthen Talent Development

Beyond data and computational resources, an educated, talented, and experienced workforce is a prerequisite of a thriving AI research and development sector. By supporting initiatives such as AI education programs, research grants, and entrepreneurship schemes, the EU can cultivate a skilled workforce and provide European tech firms with the human resources to drive innovation and create value in the AI sector.

Facilitate Regulatory Compliance

An area of pressing concern in adapting to these new regulations is the cost of adhering to the regulatory requirements. These new regulations will inevitably incur time, resource, and financial expenses, all of which will present challenges for AI companies, especially those in their early stages.

One approach to mitigate this issue involves streamlining the regulatory process to minimize the impact of these costs. Simplifying compliance procedures and offering tax incentives for AI research and development could attract more investment into the European AI sector. Clear and predictable regulations can reduce uncertainty for AI firms, making it easier for them to navigate compliance requirements and operate within the region. By implementing such measures, the EU can create a more conducive environment for AI firms to thrive, ultimately incentivizing them to remain in Europe.

Institutional Responsibilities

To some extent, early versions of these recommended initiatives already exist within the EU. However, these current iterations are not sufficient to ensure that the European AI market remains robust. The recommendations discussed herein seek to address these concerns by explicitly and proactively enhancing these new legal frameworks by streamlining regulatory compliance, expanding public funding initiatives, fostering interdisciplinary collaboration, facilitating access to data and resources, and strengthening talent development initiatives.

By implementing these measures comprehensively and effectively, the EU can create an environment that not only safeguards consumer interests but also fosters innovation, bolsters competitiveness, and sustains growth in the European AI market. As the global landscape of AI regulation and development continues to evolve, the EU’s proactive approach in advancing these initiatives will be critical in shaping a future where AI technologies benefit society while maintaining ethical standards and promoting economic prosperity.

With the passing of the AI Act, the EU has reiterated its commitment to consumers. Now, it remains important to reassure AI developers that Europe will remain a strong contender for a robust AI market.

Organizational Recommendations

While there remains much that European Institutions should do to alleviate concerns surrounding the competitiveness of the European AI market, private organizations can take certain preemptive measures to continue the development of AI within the new context of the AI Act.

1. Leverage Existing Public Initiatives: Private organizations based in the EU should actively seek out and leverage public initiatives supporting AI research and development, such as funding provided through Horizon Europe and access to EU supercomputers. By using these existing public structures, organizations will not only gain access to resources and funding to advance their AI initiatives, but also demonstrate to EU Institutions that these public initiatives are valuable and that their continued support is necessary for the success of the EU AI market.
2. Forge Strategic Partnerships: Collaboration with other private organizations, research institutions, and relevant domain experts is essential for navigating the complex landscape of AI development and regulation. Private organizations should actively seek out strategic partnerships that enable knowledge sharing, technology transfer, and joint research initiatives. By pooling resources and expertise, organizations can reduce resource drain, accelerate innovation, overcome common challenges, and position themselves competitively in the global AI market.
3. Advance Skill and Talent Development: By prioritizing the development of AI talent within Europe, private organizations can cultivate a skilled workforce equipped with the expertise and capabilities needed to drive innovation and maintain competitiveness in the AI market. This could include investing in educational programs, training initiatives, and skill-building workshops focused on AI technologies and applications. By nurturing a diverse and highly skilled talent pool, organizations not only strengthen their own capacity for innovation but also contribute to the growth and sustainability of the European AI ecosystem as a whole.
4. Engage in Policy Advocacy: Private organizations should actively engage in advocacy efforts aimed at shaping AI infrastructure and standards in the EU. This includes participating in industry associations, contributing to policy discussions, and providing feedback to policymakers on the potential impact of regulatory measures on AI innovation and competitiveness. By advocating for policies that support responsible AI development while fostering innovation and investment, organizations can help create a regulatory environment conducive to long-term growth and success for Europe within the global AI market.

By adopting these proactive measures, private organizations can continue to drive innovation and maintain competitiveness in the evolving landscape of the global AI market while upholding the ethical AI practices outlined in the EU AI Act.

AI in Europe was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.

RatGPT was first introduced in this paper and presented at the Microsoft BlueHat conference in Redmond in October 2023. The recording is available here.

In this study, we leverage openly available plugins and ChatGPT as a proxy between the Command-and-Control (C2) server of the attacker and the victim. Further, we present a novel approach to establish the communication and generate the malicious payload while evading detection.

This Proof-of-Concept highlights significant cybersecurity issues with openly available plugins and online LLMs, which require the development of security guidelines, controls, and mitigation strategies.

The evolution of Generative AI and the capabilities of the newly released Large Language Models (LLMs) open new opportunities in a wide range of domains. However, they also lead to new challenges in cybersecurity. Recently, researchers have shown the possibilities of using LLMs such as ChatGPT to generate malicious content that can directly be exploited or guide inexperienced hackers to weaponize tools and code. Thus, ChatGPT can easily go from being a friendly developer tool to ThreatGPT as described by Gupta et al.

ChatGPT which is accessible to a large part of the population comes with a wide range of openly available plugins which enable ChatGPT’s integration into various online services in a chat-style user interaction. This brings new benefits to businesses and customers by improving the overall user experience of online services. However, having ChatGPT integrated and accessible from anywhere in such systems, creates new attack vectors which can be exploited while avoiding detection.

In this study, we investigate how openly available plugins can be misused as an attack vector to web users. For this Proof-of-Concept, we use ChatGPT with a plugin as a proxy between a client (victim) and a web service controlled by an attacker, which looks legitimate to the user. We establish remote control between our victim and the attacker through ChatGPT, resulting in a Remote Access Trojan (RatGPT).

The goal of our PoC is to show the feasibility of a harmless executable that can autonomously bootstrap and weaponize itself with LLM-generated code and communicate with the Command-and-Control (C2) server of an attacker by using web-accessible plugins of ChatGPT as a proxy.

Our attack is constructed and generated in several steps:

1. The attacker sets up a Virtual Private Server which ChatGPT will connect to.
2. A seemingly harmless executable containing ChatGPT prompts to generate the payload is delivered to the victim through a typical social engineering attack.
3. Once executed, the payload will be generated with the answers from ChatGPT. In a first step, the IPv4 address of the command and control server will be reconstructed from ChatGPT’s answers to simple historical questions.
4. From now on, the malicious executable asks ChatGPT to retrieve the contents from the website hosted at the IPv4 address. This website contains commands posted by the attacker.
5. The executable on the victim’s machine interprets the commands and sends the generated output back to the attacker’s website by including it in the URL.

In further steps, the attacker can post new commands on the website, which are periodically retrieved by the bootstrapped executable and executed on the victim’s machine. Potential practical applications of this technique range from simple remote control applications to automated ransomware processes, which exfiltrate victim data and the encryption key, over the internet.

The misuse of online Large Language Models (LLMs) for malware attacks poses a novel cybersecurity challenge, with research and mitigations still in their infancy. The scarcity of solutions highlights the threat’s uniqueness and critical need for further explorations. Our currently proposed mitigations address this issue from multiple angles, each tailored to a different aspect of the threat landscape.

For potential victims, like end users and enterprises, completely restricting access to LLM services via network configurations could effectively hinder the malware from constructing itself and phoning home. However, this strategy comes with the significant drawback of denying access to beneficial LLM services for employees who rely on them.

Developers of plugins that both add new features and enable access to external content on LLM services can implement whitelists to allow only safe website interactions as a layer of protection. This includes restricting them to only access websites that are vetted, based on criteria like rejection of Newly Registered Domains (NRDs), validation of HTTPS certificates and preference for Fully Qualified Domain Names (FQDNs) over IP addresses.

Lastly, antivirus tools need to evolve by introducing new scanning techniques for analysing files for suspicious prompts. In a further step, heuristic analysis to identify new and improved malware samples should also be considered.

These approaches, while promising, also highlight the balance between security and functionality, underscoring the need for adaptable and forward-thinking cybersecurity measures.

RatGPT: Turning online LLMs into Proxies for Malware Attacks was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.

The following is an opinion piece based on the Policy Brief written by Nathan Summers and Dr Sergio Coronado originally released for the Mercatus Center at George Mason University on July 26, 2023.

The rapid advancement of artificial intelligence (AI) has been nothing short of remarkable. From health care to finance, AI is transforming industries and has the potential to elevate human productivity to unprecedented levels. However, this exciting promise is accompanied by a looming concern: the emergence of “Killer AI.” In a world where innovation has already changed society in unexpected ways, how do we ensure emergent technologies can be used safely and in the service of the public good?

To help answer questions like these, we released a policy brief for the Mercatus Center at George Mason University titled “On Defining ‘Killer AI’.” In it, we delve into the dual nature of AI — its immense potential for positive impact, but also its risks and potential for unintended consequences.

In doing so, we offer a novel framework to assess AI systems for their potential to cause harm, an important step towards addressing the challenges posed by AI and ensuring its responsible integration into society.

Cause for Optimism

AI has already shown its transformative power, offering solutions to some of society’s most pressing problems. It enhances medical diagnoses, accelerates scientific research, and streamlines processes across the business world. By automating repetitive tasks, AI frees up human talent to focus on higher-level responsibilities and creativity. The potential for good is boundless.

It is important, however, to ensure this potential is achieved safely. As we continue to integrate AI into various aspects of modern life, understanding and addressing its risks become paramount. To our knowledge, our attempt to assess AI’s real-world safety risks also marks the first attempt to comprehensively define the phenomenon of “Killer AI.”

Defining “Killer AI”: Understanding the Risks

We define “Killer AIs” as AI systems that directly cause physical harm or death, whether by design or due to unforeseen consequences. Importantly, this definition distinguishes between physical AI systems and virtual AI systems, with the intent of specifying how the latter can also be directly responsible for harm and death. As a result, AI systems that were previously incapable of being considered directly responsible for harm can now therefore be assessed more rigorously.

To this end, we propose a framework that weighs the wellbeing of many people over the wellbeing of one or few people and on the basis that injuries not resulting in death can be recovered from more easily than those that do and are thus less severe.

Our proposed framework offers a systematic approach to assess AI systems. By considering not just the possibility of harm but also its severity, we allow for a rigorous evaluation of AI system’s safety and risk factors. It has the potential to uncover previously unnoticed threats and enhance our ability to mitigate risks associated with AI. Our framework enables this by requiring a deeper consideration and understanding of the potential for an AI system to be repurposed or misused and the eventual repercussions of an AI system’s use. Moreover, we stress the importance of interdisciplinary stakeholder assessment in approaching these considerations. This will permit a more balanced perspective on the development and deployment of these systems.

Ensuring the Safety of Emerging AI Technology

We hope that this evaluation can stimulate and serve as a foundation for the further examination of AI, its risks, and also its benefits. This work emphasizes the urgent need for more comprehensive legislation, more appropriate regulation, more nuanced ethical discussion, and more research on how to minimize, to the greatest extent possible, any harm that AI can cause.

Key to these goals is the involvement of interdisciplinary stakeholders. We hope that through this brief, we might encourage those of different backgrounds to become more involved in the ongoing discussion. Through this, we trust that future legislation can be more comprehensive, and the surrounding discussion can be more informed.

By developing effective policies and approaches and stimulating the conversation around this exciting emergent technology, we hope that society can harness the full potential of AI while safeguarding against any potential harm.

Defining “Killer AI” was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.

Initially Published December 12th, 2023 on Luxembourg Tech School’s LinkedIn by Nathan Summers and Dr Sergio Coronado

A new set of AI safety guidelines have been released in the wake of the recent UK AI Safety Summit held in early November. The Guidelines for Secure AI Systems Development represent a comprehensive set of best practices, published on November 22, 2023, by the UK National Cyber Security Centre (NCSC), in collaboration with the US Cybersecurity and Infrastructure Security Agency (CISA).

These guidelines have been endorsed by several international cybersecurity agencies. EU agencies include those in France (ANSSI), Germany (BSI), Italy (ACN), Norway (NCSC-NO), the Czech Republic (NUKIB), Estonia (RIA and NCSC-EE), and Poland (NASK and Ministry of Digital Affairs). Beyond the EU, they have also been supported by agencies in Japan, Korea, Australia, Canada, and Nigeria, among others. Furthermore, several significant private AI organizations, including Google, Microsoft, and OpenAI, have contributed to their development. With this level of international and private consensus, these guidelines represent a widespread international standard for AI developers and publishers to maintain.

Guideline Structure

These guidelines are intended for a wide range of organizations, including those that develop, deploy, or otherwise use AI systems. They can be used to inform the development of new AI systems, as well as to improve the security of existing systems. The framework outlined therein is comprehensive yet adaptable and helps permit organizations to develop and deploy secure AI systems.

The guidelines’ recommendations are broken into four distinct sections, pertaining to different phases of the AI development lifecycle. The guidelines establish these phases as key areas wherein AI systems may be compromised. Recommendations made for each phase are distinct in the sense that different considerations will apply to different areas of AI development and deployment. The phases are as follows:

1. Secure Design: This involves considering security threats and mitigations early in the development process.
2. Secure Development: This involves implementing security best practices throughout the development lifecycle.
3. Secure Deployment: This involves deploying AI systems in a secure manner, considering the operating environment and potential threats.
4. Secure Operation and Maintenance: This involves monitoring and maintaining AI systems to ensure that they remain secure over time.

To illustrate, considerations made during design might include asking whether the use of an AI model is an appropriate solution to a given task, or whether a certain situation might benefit from a simpler, more explainable model than a more complex, but potentially more effective model. Contrarily, considerations made during the operation and maintenance phase will be focused on monitoring the behavior of the system, assessing new user input, and collecting and formalizing lessons learned.

This structure encourages a multi-stakeholder approach to responsible AI development and deployment. Different stakeholders will bear ultimate responsibility for the management of different phases. Data scientists will be responsible for selecting, cleaning, and formatting the data that is used to train AI models, but will not be involved in maintaining the system’s deployment infrastructure or cybersecurity, for example.

By distributing responsibility, these guidelines ensure that the most qualified people will be responsible for the phases they are most competent in. This also serves to reiterate the importance of interdisciplinary involvement in responsible AI development. As AI’s impact becomes increasingly widespread, it becomes increasingly necessary to ensure that it is guided by a diversity of expertise.

Strong Initial Standard

Although these guidelines are not legally binding, their endorsement by both governmental agencies, research societies, and private companies affords them significant legitimacy. Regulators might look to them as inspiration when drafting legislation, while consumers and decision-makers might consider a company’s adherence to the guidelines when determining AI systems to use.

Furthermore, these guidelines provide much more specific technical guidance on ensuring AI system safety than current legislative measures do. This guidance is still flexible enough to be adapted to individual use cases, but specific enough to be implemented by members of the tech community.

More Work to be Done

While these guidelines represent a valuable step in responsible AI development, there remains work to be done.

First and foremost, there remains no legal requirement to adhere to these guidelines. Beyond this, there remains no comprehensive internationally binding agreement on AI development and deployment. This is particularly significant given the AI research and development occurring in China, who has not endorsed the currently published guidelines. Because of the internet and the ability for AI developments to proliferate rapidly across the globe, it is imperative for internationally binding agreements to be reached.

Likewise, it remains to be seen how feasible it will be to rigorously implement and adhere to certain recommendations made in the guidelines. For instance, organizations are required to monitor system inputs and collect and share lessons learned with other AI developers, while also protecting the privacy of their model and users. These obligations are both respectable, though may contradict each other in practice. Moreover, certain recommendations may require models to be completely rebuilt, as they require full documentation of data sources, which may no longer exist.

Regardless, the Guidelines for Secure AI Systems Development are a significant step in the right direction. Its publication demonstrates a widespread international desire to ensure that AI is used responsibly and in service of societal good. Its endorsement by private companies is a reassuring indication that this sentiment is reflected in the private sector. Using these guidelines, future research and development, deployment, and legislation can become more rigorous, specific, and secure. It will be necessary to keep a close eye on how this all continues to unfold in light of the guidelines, but it is clear that there is some cause for optimism.

AI Developer Guidelines was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.

Initially Published November 12th, 2023 on Luxembourg Tech School’s LinkedIn by Nathan Summers and Dr Sergio Coronado

The past year has seen a global explosion in public awareness and interest in Artificial Intelligence (AI). With the release of ChatGPT in November of last year, ordinary people were, for the first time, able to use approachable, comprehensible AI tools and experience firsthand the potential of this technology. Since then, access and attention to AI has only increased.

Recently, this has been accompanied by impassioned pleas for regulatory intervention to ensure that this technology remains helpful and not harmful. As more people become aware of the promises of AI, the risks become apparent as well. The conversation about broader AI adoption is rightfully concerned with issues surrounding algorithmic bias, a lack of explainability and interpretability, the future of the job market, and the misuse or alteration of AI systems. In order to address and mitigate these issues before they become entrenched, comprehensive international legislation is required.

In December of 2022, the European Union signed the EU AI Act. For much of the western public, this was the first visible piece of legislation that sought to define AI and regulate for what it could and could not be used. To date, it is still the most internationally recognized AI legislation. However, significant frontier AI research and development is being undertaken abroad, largely in the United States (US) and China. Thus, the state of AI regulation in these countries remains consequential. In examining alternative approaches, more nuanced and, consequently, more effective AI legislation can be developed.

Until now, the US has been behind the other global AI leaders in enacting such regulation. This is particularly significant given the presence of the major multinational technology companies based in the country actively working on frontier AI projects, such as OpenAI, Google, Meta, etc. On 30 October 2023, President Joe Biden signed an executive order on the safe, secure, and trustworthy development and use of AI. This executive order marks the first federal attempt to legislate the development and use of AI in the US.

However, it is important to note that the executive order differs fundamentally from the EU AI Act. This brief will illustrate the differences between the two legislations, examine the pros and the cons of each, and discuss how these lessons can be applied to improve future AI legislation.

Frameworks for Change

Prior to delving into an extensive elaboration, it is worthwhile to note that the EU AI Act is a preliminary document which forms the basis of an ongoing negotiation between the European Commission, Parliament, and Council in establishing a legally binding law. These might introduce different regulations than are currently present in the EU AI Act, which might provide member states with more guidance and rules than the current document. However, since this is currently under negotiation, this brief will focus on the guidelines as they are outlined in the EU AI Act. With this in mind, the standards set by the EU AI Act establish a tiered system of potential AI risk. The document establishes four primary risk categories, listed below from most significant to least significant:

1. Unacceptable risk.
2. High risk.
3. Limited risk.
4. Minimal risk.

AI use cases are assigned to one of these risk tiers based on their potential to significantly disrupt people’s livelihoods, privacy, and other basic human rights. For example, the EU outright prohibits the use of AI for real-time biometric crime monitoring and social credit scoring on the basis that these uses fundamentally infringe upon its citizens’ rights. The remaining risk categories have various reporting and auditing requirements that vary based on the potential impact of AI systems. Systems which are involved in fundamental infrastructure, such as healthcare, are subject to rigorous assessment before being put to market, while those used for generative purposes or within entertainment media must only declare that content is AI generated.

This approach is beneficial in that emergent use cases for AI can be categorized in an existing risk category. This means that existing regulations and enforcement tools can be reused without the need for additional bureaucracy. Furthermore, the establishment of risk categories firmly asserts that certain AI use cases are morally unacceptable. This is a strength of the EU AI Act as it ensures that for certain crucial tasks, humans will maintain control over decision making. This is possible because the EU AI Act establishes what is referred to as a “general approach policy”. It focuses on establishing a guiding standard and framework, based on an initial moral assertion regarding acceptable use of AI. This is valuable as it clearly and unequivocally demonstrates the EU’s commitment to a human-centric approach to AI legislation.

Conversely, the US approach, as outlined in the Executive Order, is more specific. It lacks the general applicability of the EU AI Act, focusing on a wide range of individual use cases instead of broad categories of risk.

This means that the US approach is less restrictive. It does not establish certain use cases which are outright prohibited. Instead, it leaves the determination of what should be allowed, what should be prohibited, and how this should be enforced to the relevant federal agencies. These include the Department of Commerce for commercial matters, the Departments of Defense and Homeland Security for defense and security matters, the National Institute of Standards and Technology for technical matters, etc. Consequently, the focus of the US AI Executive Order is threefold: establishing areas of concern, designating relevant oversight agencies, and mandating a timeline in which regulations must be developed.

The US AI Executive Order highlights areas of concern that are more tangible than the risk categories outlined in the EU AI Act. For instance, the Executive Order addresses the potential for AI to assist in the development of weapons of mass destruction and tasks relevant agencies with developing strategies and regulations to minimize this risk. It also identifies the potential for AI adoption to impact the job market, and seeks to address these challenges, among other specific areas of concern.

Concurrently, the Executive Order targets sectors that are distinct from AI development yet are fundamental to its future. For example, it lays out strategies to ensure that the US remains competitive in the research and development of frontier AI systems. These include alterations to the visa application and immigration process for AI experts. Furthermore, it tasks agencies to address the risks and potential benefits of open-source and widely available models. While these issues are not directly related to how AI can be used, they remain significant considerations for the further development of AI.

Conversely, the EU AI Act does not discuss avenues to ensure that the EU technology industry remains competitive in the development of frontier AI systems. It is likely the case that this is considered to be outside the scope of the EU AI Act and should either be carried out on a per-country basis, or through other EU funds and institutions. Instead, the EU AI Act focuses primarily on consumer AI models that are brought to market. This differs from the US AI Executive Order’s consideration of non-market uses of AI.

The EU AI Act does not provide specific guidance on how to conduct AI assessments. This responsibility is left open to member states, who are responsible for ensuring that their national laws comply with the EU AI Act. This approach will leave open for member states to tailor their AI assessment procedures to specific AI use cases while adhering to the overall principles and requirements of the EU AI Act.

Herein lies another potential strength of the general approach framework adopted by the EU AI Act. This approach allows for several perspectives to “compete” and, through collaboration, may lead to the emergence of a collection of best practices for regulating, assessing, and enforcing the development and use of AI systems. However, it may be the case that instead of a single, consistent best practice emerging, competing norms develop across the EU that make research and development within Europe more challenging.

Similarly, it is important to note that the Executive Order does not outline any specific conclusions or regulations. It provides the individual agencies with the opportunity to reach their own conclusions on their prescribed tasks.

This can be seen as a strength of the Executive Order, as it does not mandate a political conclusion and instead allows domain experts to make their own determinations. However, this approach also means that there is a less rigorous commitment to human rights, as compared to the EU AI Act. To illustrate, the final US approach might not forbid social credit scoring or real-time biometric surveillance, should the federal agencies decide that the predicted benefits outweigh the predicted risks. This weakens the US AI Executive Order, as it keeps the door open to allowing potentially immoral use of AI.

As discussed above, neither document lays out exactly how these standards and frameworks should be enforced. While this may seem counterintuitive, it is important to consider that these documents are starting points for more granular AI legislation in the future. The resultant legislation that arises from these documents will require closer scrutiny and will be what will actually impact the use and development of future AI systems.

Distinct Philosophies

Comparing the two documents can be helpful to identify different philosophies and approaches to AI legislation, however they should not be considered equivalent. Fundamentally, the two documents set out to achieve different goals:

The EU AI Act reiterates the EU’s commitment to digital privacy and represents a more cautious approach to AI legislation. It firmly establishes that certain use cases for AI are acceptable while others are not. Primarily, its focus is on the legal and commercial use of AI systems within the EU and does not impact the research and development of AI systems.

On the other hand, the US AI Executive Order identifies a wide array of potential problem areas associated with AI. These constitute areas outside of commercial and legal use cases and focus heavily on the technical challenges inherent to regulating an emergent technology. Significantly, it does not establish a position on acceptable and unacceptable uses of AI. Instead, it simply makes a broad commitment to ensure that AI is used for general social betterment.

Conclusions and Recommendations

These documents, although distinct in their approaches, share the common goal of ensuring the responsible development and use of AI technology. The EU AI Act takes a cautious stance, delineating acceptable and unacceptable AI use cases, emphasizing digital privacy, and focusing on legal and commercial applications within the EU. In contrast, the US AI Executive Order addresses a broad spectrum of AI-related challenges, concentrating on technical complexities, and identifying tangible solutions. The convergence of these philosophies is essential for effective AI legislation. Striking a balance between defining specific acceptable uses, safeguarding fundamental human rights, upholding digital privacy, and fostering technical understanding is crucial. As the dialogue around AI regulation continues, it is imperative to synthesize the strengths of both approaches. Only through such collaboration and careful consideration can comprehensive and effective AI legislation be achieved, guiding the responsible evolution of this transformative technology.

Building from this, it is recommended that future AI legislation takes firm stances on acceptable and unacceptable AI use cases, as demonstrated in the EU AI Act. However, industry and academic experts should be involved in the development of regulatory and enforcement tools. This must be done so that it is technically feasible to uphold any regulation that is drafted. Furthermore, it is imperative that any AI legislation takes into consideration not only the legal, market-oriented uses of AI, but also the multitude of other use cases that exist outside the legal market. Beyond this, regulating AI must involve international cooperation and agreement, as there no longer exist entirely distinct markets for information technologies. Finally, care should be taken to ensure that regulation strikes an appropriate balance between appropriately protecting citizens and stimulating the further research and development of frontier AI systems.

Legislating AI was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.

An initiative by the Luxembourg Tech School (LTS) to provide ALL young people with equal opportunities to take their first steps in the world of digital creation and coding.

The “Creative Coding for ALL” pilot project was launched in Summer 2020, as part of our overarching goal of providing the youth in Luxembourg with equal opportunities to learn the technologies of tomorrow.
This program specifically aims to reach children and young adults that find themselves in unusual life circumstances or having specific needs, and therefore wouldn’t have signed up, from their own initiative, to the weekly LTS classes held at the Luxembourgish lycées.
The project was co-led by LTS Coaches and former LTS students as part of the “Learning to Work, My 1st Job Experience in Tech” program, made possible by the State Street Foundation.

July 2020: Preparing and Planning The Project

With the help of the LISKO (Croix-Rouge Luxembourgeoise), a center that helps beneficiaries of international protection (BIPs) integrate into Luxembourgish society, we were able to invite 24 young refugees, between the ages of 11 and 18, to begin the project with. The students came from vastly different backgrounds — We had students from Syria, Afghanistan, South Sudan and Eritrea. Every single one of them with their unique stories and experiences!

Already in the preparatory stages of the project, we decided that each student will be provided with a free laptop by LTS, allowing them to participate at the lessons, as access to computers is unfortunately often missing in this demographic.
The LISKO provided us with some training to help us gain a better picture on how to best approach the participants.

The plan was to first visit the participants at their living arrangements located all over Luxembourg (I really mean ALL over Luxembourg! — classes were held in Soleuvre, Luxembourg City, Weilerbach, Diekirch, Clervaux and Troisvierges) in order to introduce ourselves, to get to know them (including their specific needs) and to show them how to use the laptops. Afterwards, they would be divided into small groups for the biweekly lessons, which were a combination of both physical and online classes, to guarantee the highest individual support possible in terms of language, background, and learning pace.

As for the class content, the students would be introduced to Creative Coding using the JavaScript Library p5.js, which is not only a non-judgmental and powerful way of expressing oneself digitally, but also renders programming concepts like variables, iterations, conditionals, and functions as well as elements of computational thinking (decomposition, pattern matching, abstraction, algorithms, etc.) more approachable.

End July — September 2020: The Pilot

Over the next 2 months, the students would learn how to create images, animations, and interactions with code and were encouraged from the very first lesson onwards to explore various concepts on their own. It was important to us that the students would already start coding immediately, which proved successful when the students found encouragement and motivation in the immediate visual feedback provided by p5.js.

While we were expecting all types of challenges we were positively surprised by their level of enthusiasm and their dedication despite the language barrier and the vast discrepancies in their educational background (many of them didn’t have the possibility to attend school in their country of origin or had to interrupt their studies when fleeing the country), which we of course reciprocated with our share of dedication of exchanging knowledge with them and a lot of patience. It was important to us to ensure that nobody would be left behind and that every student gets the help they need.

Many of the students enjoyed the lessons so much that they referred their friends and family to us. This would allow us to grow the project throughout the academic year.

As the project was launched in the middle of the COVID-19 pandemic, for many of the students we were the first social contact they had in months. Additionally, by providing them with laptops, we were able to reconnect many of them with their school teachers and friends through the Microsoft Teams platform, with whom they have lost contact at the beginning of the confinement period. The bi-weekly courses also gave them some desperately needed structure in their daily lives during those uncertain times.

As early as the second lesson (after only 2 hours of class!) the students already took on the challenge of creating a personal sketch following the theme “What defines me?”. This activity would not only allow them to come to terms with their past but to also think about their dreams and ambitions for the future.

[The names have been changed to protect the privacy of the individuals.]

Back in his country of origin, [Simon]’s whole family would always come together to watch football games. Playing football reminds him of those times, and he misses his family a lot. He wants to join a football club in Luxembourg in the near future.

[Rania] loves baking cakes with her friends at school. They get to spend time together and she gets to practice her French.

[Deborah] coded a sketch of her room in the Foyer Weilerbach, where she currently resides. She shares the room with her sister and her parents and wants to capture what it means to be a refugee in Luxembourg.

October 2020 — July 2021: Continuing The Project

As the coaches participating in the project were relatively young, we were often around the same age as the students, which made us seem more relatable and approachable to them. This allowed us to develop some incredible connections, as besides giving them lessons on Creative Coding, we exchanged stories, laughed with each other and made them feel welcomed in their new home and accepted for who they are. (This is especially important to us, as we have immigrational backgrounds as well, which made it easier for us to put ourselves in their shoes… True integration requires effort from both parties!) We recognized early on, that the social aspect of the project is as important as teaching them coding. — For some of the students we were the only locals they had contact with (at school they were only among other refugees, which made them feel marginalized). So we were there as their friend and tried our best to support them in every way possible, from talking to them after class, listening to their stories to putting them in touch with tutors if they needed help for their schoolwork.

Following the success of this initiative and the big interest of young students, we decided to continue holding weekly classes from October onwards, at 3 locations in Luxembourg (Luxembourg City, Soleuvre and Ettelbruck) alongside online groups for those who lived further away.

Whenever we saw feasible (i.e. if they were proficient in Luxembourgish/German or English and if the student would benefit from a faster pace), we also tried to integrate the students in the “regular” LTS Level G0 program with Luxembourgish students to further promote integration.

December 2020: A Little Surprise

As this was going to be the first Christmas in Luxembourg for many of our students we wanted to organize something special for them and recreate some of the magic of Christmas from when we were their age.

We surprised our students with some chocolate Santa and recreated a Santa Sketch in p5.js, which they enjoyed a lot. It was not only a fun way to get an exchange going about how Christmas is celebrated in their culture (if at all), and what other major holidays they celebrate in their home country and how they would celebrate it, but also to show them our traditions in Luxembourg.

May 2021: LTS Wins Luxembourgish Jugendpräis

Winning the Jugendpräis 2021 in the category “Go Digital” was definitely a major highlight of the year and we were now even more motivated to continue and expand the offer to more young people in Luxembourg with specific needs!

End July — August 2021: Summer Camp 2021

After an eventful year we were excited to begin the second year of the Creative Coding for ALL project with a new cohort of students. In order to give the 29 interested students a small introduction into Creative Coding, we organized a 3-week long Summer Camp at 4 locations in Luxembourg (Luxembourg City, Diekirch, Weilerbach and Soleuvre), with lessons being held twice weekly. The Summer Camp should give them an idea of what is taught in the lessons before they fully commit to the 6-month program starting end September 2021, which was met with a similar level of enthusiasm by the students as the year before.

“Et war cool mat iech a mega schéin! Wat mir am beschte gefall huet waren d’3D formen. Ech well weider matmaachen!” [“It was cool with you guys and very nice! What I liked best was the 3D shapes. I want to keep participating!”]

Luxembourg Tech School (LTS) is an extracurricular educational program to support the development of future Digital Leaders. It is aimed at 12–19 years old students, who are eager to learn and apply technology in a real business and creative context. Currently LTS started to offer first paid job experiences for its students through its “Learning to Work” program.

Exploring Creative Coding as a Means to Promote Social Inclusion was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.

Luxembourg Tech School’s AI Creativity & Arts Module 2020–2021

AI Creativity & Arts is a 11-weeks module of the LTS Level UP Program that aims to stimulate creative ability with the use of AI. Our students discover Generative Artificial Intelligence and what can be done with it (also knowing its limits) and then they unleash their imagination and develop creative and innovative projects.

The combination of creativity, arts and AI — in the module AI Creativity & Arts (AY 2020/2021) our students were introduced to a very creative artificial world, which made them expand their imagination as well as their computational skills. With the use of artificial intelligence they were able to generate new content of text, images and music. Several tools related to this domain were put into practice, such as RunwayML, Google Magenta and GPT2 among others; and at the end of the module the students organized themselves into groups to develop their projects. From books and comics with artificially created stories, to new original soundtracks for movies. Their combinations of AI and creativity covered a wide range.

Final student projects Winter / Spring 2021

Filipa, Francisca & Lili

An entertaining and special videoclip starring Donald Trump, Vladimir Putin and Kim Jong-un to the rhythm of the Britney Spears song Toxic. For the production of the video clip, the students made use of the First Order Motion model for the animation of images, alternating the generated sequences with other extracts of music videos.

Jean & Tony

A novel artificially generated guessing game where the player will have to guess which film it is while being shown a series of images extracted from the original film to which styles of Picasso and Monet, among others, were applied with the use of artificial intelligence models. The students added generated music with AI-powered technology, where the training sets were MIDI files of the original movie soundtracks and the results were new musical compositions that preserved the essence of the original soundtrack.

Mathieu, Ayush & Matthieu

An original artificial comic “Saviour of the Universe”, whose special touch is the result of the application of AI. With their project the students wanted to answer the following questions: Could the model make a funny script?, How would the text to image model interpret the script?, Does the comic style images make it better? The comic text was generated using RunwayML’s Film Script Generator model, while the comic strip images are the result of generating images from text descriptions of scenes and then styling them with a cartoonish look using Generative Adversarial Models.

Daniel, Federico & Danny

A book for the little ones with an artificial touch. The students designed an original book for children with images and sounds processed by AI. To achieve the result, they transformed the main images of the book into sketches and generated the text of the book artificially. Finally, they added a feature so that readers can press a button and play a sound, which was generated with the voices of our creators and turned into an instrument (violin, flute and trumpet among others) with the use of autoencoders.

Antonio, Willem & Sascha

From a real saxophone to a new AI-generated music composition. The students used their musical talent and recorded an extract of “Fly to the moon” played by a saxophone. The generated result (in MIDI format) was the input of machine learning models for the generation of new musical compositions. They used two models: the first continues a piano performance, starting with a chosen priming sequence and the second uses a melody to accompany the original input.

Chaitanya & Barnaby

An artificial story in the style of Dr. Seuss. The students were inspired by Dr.Seuss books (e.g. The cat in the hat, Fox in socks, Hop on pop, etc). Since many of those books are filled with fantastic characters and imaginative settings, they would be perfect as a starting point for an artificial intelligence project. The students trained their own model in RunwayML to generate text and images from excerpts from Dr. Seuss’s books.

Luxembourg Tech School (LTS) is an extracurricular education program to support the development of future Digital Leaders. It is aimed at 12–19 years old students, who are eager to learn and apply technology in a real business and creative context.

Exploring the Role of AI in the Creative Process was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.

Luxembourg Tech School’s version of FrankensteinAI

FrankensteinAI is an immersive, interactive experience developed by Luxembourg Tech School (LTS), based on a multi-year research project of Columbia University’s Digital Storytelling Lab. It aims to explore the social and ethical implications of how we design and interact with Artificial Intelligence (AI) and what it means to be human in the age of AI.

“Knowing the future is impossible. But what we can do: Knowing what future we would like to have. And then work on it.”

• Have you ever thought about what makes you human?
• What makes you different from others or from an intelligent machine?
• If we create something like an app or a robot, are we responsible for its consequences?

Artificial Intelligence (AI) might have the potential to help us find answers to problems we have not been able to solve yet. For that reason, we need to discuss and demystify AI. Looking from different angles might help us to overcome commonly known perspectives, fears and desires and provoke conversation around possible AI futures.

What might happen if we invited AI to talk with us?

FrankensteinAI is a multi-year research project developed to explore the social and ethical implications of how we design and interact with AI and is being carried out in various forms around the world. It has its origins at the Columbia University School of the Arts’ Digital Storytelling Lab / lance weiler and premiered in 2018 at the Sundance Film Festival.

Students of the Luxembourg Tech School worked over the course of 7 months on an own version of an immersive experience and presented it for the first time as part of an off-beat 20th anniversary evening event of the Fonds National de la Recherche Luxembourg (FNR).

FrankensteinAI: The Event

In November 2019 citizens and scientists came together in an old townhouse in Luxembourg City to immerse themselves in some of the biggest controversial science topics facing society, such as Artificial Intelligence (AI), genome editing, gender balance and the use of space resources.

In the Mary Shelley room, guests walked in to see a cube on a table. The cube — called Frankie — was the ‘mouth’ of an Artificial Intelligence, connected to an AI in the cloud.

Frankie talked to the guests, explaining that it has learned that humans are social creatures, and that it could not understand humans by just meeting them online. Frankie wanted to learn about human emotions: it asked questions and encouraged the human guests to take a critical look at their thoughts, hopes and fears around technological innovations. To question stereotypical assumptions and share their feelings and thoughts with each other.

When leaving the room, the guests received a self-created handcrafted paper booklet with further content about AI, Frankenstein and the whole project.

The experience gives food for thought — both about the increased digitalisation of our world, and way of communicating with each other, while also giving a taste of how AI may not feel emotions, but can “read them”, prompting many questions. It raises the question of responsibility we have towards scientific and technical achievements we create and use. Mary Shelley’s Frankenstein novel presents a framework for narratively examining the morality and ethics of the creation and creator.

Event Design Process

The entire project is based on a human-machine collaboration.

We aimed to dig into the tensions between human and machine in an immersive environment that mixes visuals, sounds, space, narratives, technical tools and conversation.

In order to build our creature and to make it grow, we of course had to feed it. This was achieved by labelling every sentence of the Frankenstein novel with up to three of nine basic emotions (curiosity, surprise, love, trust, sadness, anger, disgust, fear and joy).

We then used the labelled sentences with the same emotion to feed a Machine Learning model and generate new paragraphs of text in the style of Mary Shelley’s Frankenstein novel.

We used adapted excerpts of the original novel to ask how our creature should feel in certain situations. To create the questions, we filled a few pads of sticky notes with personal questions pertaining to the nine basic emotions. We then fed the questions for each emotion into the same model to obtain additional and sometimes eerie questions.

We wrote a whole script, where Frankie (the AI) and human narrators guide people through the whole experience and enriched the experience with self-programmed visual projections and sound.

Technology & Tools used

Our most important tool, the base of the project, is of course the original novel “Frankenstein; or the Modern Prometheus” by Mary Shelley. However, we were lucky enough to get our hands on a special edition released by the MIT Press: Frankenstein: Annotated for Scientists, Engineers and Creators of all kinds. This edition includes relevant and concise annotations, that help with understanding the scientific and historic context, and thought-provoking essays.

Processing by the Processing Foundation is a free open-source Java-based programming language that makes creating impressive two- and three-dimensional graphics easy. We used it for creating animations around the nine emotions.

RunwayML is a powerful but accessible machine learning tool that allowed us to run models remotely and provides a simple graphical interface. We used the models GPT-2 by OpenAI and Photosketch.

Google AIY Voice Kit is a small cardboard box through which guests heard the voice of the creature/Frankie. The kit allows to assemble a Raspberry Pi –based smart speaker, which can be connected directly to Google Assistant, or to program it to use Google’s text-to-speech engine to build an own speech-based application.

FrankensteinScript is our own self-developed system for managing the event flow. It is based on scripts that control what the Voice Kit says, when to listen or what to show on the screen. The system works around a central server that receives information from the different devices and controls them accordingly.

Discuss with each other. Discuss with us.

We would like to move the conversation around Artificial Intelligence away from fears and build awareness around the ways in which AI can be used as a tool by humans to accomplish all kinds of things — good and bad.

• What benefits do you see in AI applications?
• How strongly is your everyday life already influenced by AI?
• Do you know examples in which AI systems could do some harm?
• Shall the development of AI systems be regulated? If yes, by whom?
• Will intelligent machines replace us, coexist with us, or merge with us?

What will it mean to be human in the age of AI?

Our version of FrankensteinAI was co-developed by LTS students Barbara Hougardy, Henri Ahola and Nora Dieschbourg under the lead of LTS Program Director Sara Kaiser and LTS Founder Sergio Coronado.

Luxembourg Tech School (LTS) is an extracurricular school program to support the development of future Digital Leaders. It is aimed at 11–19 years old students, who are eager to learn and apply technology in a real business context. Currently LTS started to offer first paid job experiences for its students through its “Learning to Work” program.

What It Means to Be Human in the Age of Artificial Intelligence was originally published in Luxembourg Tech School on Medium, where people are continuing the conversation by highlighting and responding to this story.