Researchers have demonstrated that putting words in ASCII art can cause LLMs—GPT-3.5, GPT-4, Gemini, Claude, and Llama2—to ignore their safety instructions. Research paper. This article has been indexed from Schneier on Security Read the original article: Jailbreaking LLMs with ASCII…
Tag: Schneier on Security
Using LLMs to Unredact Text
Initial results in using LLMs to unredact text based on the size of the individual-word redaction rectangles. This feels like something that a specialized ML system could be trained on. This article has been indexed from Schneier on Security Read…
Friday Squid Blogging: New Plant Looks Like a Squid
Newly discovered plant looks like a squid. And it’s super weird: The plant, which grows to 3 centimetres tall and 2 centimetres wide, emerges to the surface for as little as a week each year. It belongs to a group…
Essays from the Second IWORD
The Ash Center has posted a series of twelve essays stemming from the Second Interdisciplinary Workshop on Reimagining Democracy (IWORD 2023). Aviv Ovadya, Democracy as Approximation: A Primer for “AI for Democracy” Innovators Kathryn Peters, Permission and Participation Claudia Chwalisz,…
A Taxonomy of Prompt Injection Attacks
Researchers ran a global prompt hacking competition, and have documented the results in a paper that both gives a lot of good examples and tries to organize a taxonomy of effective prompt injection strategies. It seems as if the most…
How Public AI Can Strengthen Democracy
With the world’s focus turning to misinformation, manipulation, and outright propaganda ahead of the 2024 U.S. presidential election, we know that democracy has an AI problem. But we’re learning that AI has a democracy problem, too. Both challenges must be…
Surveillance through Push Notifications
The Washington Post is reporting on the FBI’s increasing use of push notification data—”push tokens”—to identify people. The police can request this data from companies like Apple and Google without a warrant. The investigative technique goes back years. Court orders…
The Insecurity of Video Doorbells
Consumer Reports has analyzed a bunch of popular Internet-connected video doorbells. Their security is terrible. First, these doorbells expose your home IP address and WiFi network name to the internet without encryption, potentially opening your home network to online criminals.…
Friday Squid Blogging: New Extinct Species of Vampire Squid Discovered
Paleontologists have discovered a 183-million-year-old species of vampire squid. Prior research suggests that the vampyromorph lived in the shallows off an island that once existed in what is now the heart of the European mainland. The research team believes that…
NIST Cybersecurity Framework 2.0
NIST has released version 2.0 of the Cybersecurity Framework: The CSF 2.0, which supports implementation of the National Cybersecurity Strategy, has an expanded scope that goes beyond protecting critical infrastructure, such as hospitals and power plants, to all organizations in…
How the “Frontier” Became the Slogan of Uncontrolled AI
Artificial intelligence (AI) has been billed as the next frontier of humanity: the newly available expanse whose exploration will drive the next era of growth, wealth, and human flourishing. It’s a scary metaphor. Throughout American history, the drive for expansion…
A Cyber Insurance Backstop
In the first week of January, the pharmaceutical giant Merck quietly settled its years-long lawsuit over whether or not its property and casualty insurers would cover a $700 million claim filed after the devastating NotPetya cyberattack in 2017. The malware…
China Surveillance Company Hacked
Last week, someone posted something like 570 files, images and chat logs from a Chinese company called I-Soon. I-Soon sells hacking and espionage services to Chinese national and local government. Lots of details in the news articles. These aren’t details…
AIs Hacking Websites
New research: LLM Agents can Autonomously Hack Websites Abstract: In recent years, large language models (LLMs) have become increasingly capable and can now interact with tools (i.e., call functions), read documents, and recursively call themselves. As a result, these LLMs…
New Image/Video Prompt Injection Attacks
Simon Willison has been playing with the video processing capabilities of the new Gemini Pro 1.5 model from Google, and it’s really impressive. Which means a lot of scary new video prompt injection attacks. And remember, given the current state…
Details of a Phone Scam
First-person account of someone who fell for a scam, that started as a fake Amazon service rep and ended with a fake CIA agent, and lost $50,000 cash. And this is not a naive or stupid person. The details are…
Microsoft Is Spying on Users of Its AI Tools
Microsoft announced that it caught Chinese, Russian, and Iranian hackers using its AI tools—presumably coding tools—to improve their hacking abilities. From their report: In collaboration with OpenAI, we are sharing threat intelligence showing detected state affiliated adversaries—tracked as Forest Blizzard,…
EU Court of Human Rights Rejects Encryption Backdoors
The European Court of Human Rights has ruled that breaking end-to-end encryption by adding backdoors violates human rights: Seemingly most critically, the [Russian] government told the ECHR that any intrusion on private lives resulting from decrypting messages was “necessary” to…
Friday Squid Blogging: Vegan Squid-Ink Pasta
It uses black beans for color and seaweed for flavor. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here. This article…
On the Insecurity of Software Bloat
Good essay on software bloat and the insecurities it causes. The world ships too much code, most of it by third parties, sometimes unintended, most of it uninspected. Because of this, there is a huge attack surface full of mediocre…
Upcoming Speaking Engagements
This is a current list of where and when I am scheduled to speak: I’m speaking at the Munich Security Conference (MSC) 2024 in Munich, Germany, on Friday, February 16, 2024. I’m giving a keynote at a symposium on “AI…
Improving the Cryptanalysis of Lattice-Based Public-Key Algorithms
The winner of the Best Paper Award at Crypto this year was a significant improvement to lattice-based cryptanalysis. This is important, because a bunch of NIST’s post-quantum options base their security on lattice problems. I worry about standardizing on post-quantum…
A Hacker’s Mind is Out in Paperback
The paperback version of A Hacker’s Mind has just been published. It’s the same book, only a cheaper format. But—and this is the real reason I am posting this—Amazon has significantly discounted the hardcover to $15 to get rid of…
Molly White Reviews Blockchain Book
Molly White—of “Web3 is Going Just Great” fame—reviews Chris Dixon’s blockchain solutions book: Read Write Own: In fact, throughout the entire book, Dixon fails to identify a single blockchain project that has successfully provided a non-speculative service at any kind…
On Passkey Usability
Matt Burgess tries to only use passkeys. The results are mixed. This article has been indexed from Schneier on Security Read the original article: On Passkey Usability
Friday Squid Blogging: A Penguin Named “Squid”
Amusing story about a penguin named “Squid.” As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here. This article has been indexed…
No, Toothbrushes Were Not Used in a Massive DDoS Attack
The widely reported story last week that 1.5 million smart toothbrushes were hacked and used in a DDoS attack is false. Near as I can tell, a German reporter talking to someone at Fortinet got it wrong, and then everyone…
On Software Liabilities
Over on Lawfare, Jim Dempsey published a really interesting proposal for software liability: “Standard for Software Liability: Focus on the Product for Liability, Focus on the Process for Safe Harbor.” Section 1 of this paper sets the stage by briefly…
Teaching LLMs to Be Deceptive
Interesting research: “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training“: Abstract: Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity.…
Documents about the NSA’s Banning of Furby Toys in the 1990s
Via a FOIA request, we have documents from the NSA about their banning of Furby toys. This article has been indexed from Schneier on Security Read the original article: Documents about the NSA’s Banning of Furby Toys in the 1990s
Deepfake Fraud
A deepfake video conference call—with everyone else on the call a fake—fooled a finance worker into sending $25M to the criminals’ account. This article has been indexed from Schneier on Security Read the original article: Deepfake Fraud
David Kahn
David Kahn has died. His groundbreaking book, The Codebreakers was the first serious book I read about codebreaking, and one of the primary reasons I entered this field. He will be missed. This article has been indexed from Schneier on…
New Images of Colossus Released
GCHQ has released new images of the WWII Colossus code-breaking computer, celebrating the machine’s eightieth anniversary (birthday?). News article. This article has been indexed from Schneier on Security Read the original article: New Images of Colossus Released
NSA Buying Bulk Surveillance Data on Americans without a Warrant
It finally admitted to buying bulk data on Americans from data brokers, in response to a query by Senator Weyden. This is almost certainly illegal, although the NSA maintains that it is legal until it’s told otherwise. Some news articles.…
Microsoft Executives Hacked
Microsoft is reporting that a Russian intelligence agency—the same one responsible for SolarWinds—accessed the email system of the company’s executives. Beginning in late November 2023, the threat actor used a password spray attack to compromise a legacy non-production test tenant…
Friday Squid Blogging: Footage of Black-Eyed Squid Brooding Her Eggs
Amazing footage of a black-eyed squid (Gonatus onyx) carrying thousands of eggs. They tend to hang out about 6,200 feet below sea level. As usual, you can also use this squid post to talk about the security stories in the…
Chatbots and Human Conversation
For most of history, communicating with a computer has not been like communicating with a person. In their earliest years, computers required carefully constructed instructions, delivered through punch cards; then came a command-line interface, followed by menus and options and…
Friday Squid Blogging: New Foods from Squid Fins
We only eat about half of a squid, ignoring the fins. A group of researchers is working to change that. As usual, you can also use this squid post to talk about the security stories in the news that I…
Zelle Is Using My Name and Voice without My Consent
Okay, so this is weird. Zelle has been using my name, and my voice, in audio podcast ads—without my permission. At least, I think it is without my permission. It’s possible that I gave some sort of blanket permission when…
Canadian Citizen Gets Phone Back from Police
After 175 million failed password guesses, a judge rules that the Canadian police must return a suspect’s phone. [Judge] Carter said the investigation can continue without the phones, and he noted that Ottawa police have made a formal request to…
Second Interdisciplinary Workshop on Reimagining Democracy
Last month, I convened the Second Interdisciplinary Workshop on Reimagining Democracy (IWORD 2023) at the Harvard Kennedy School Ash Center. As with IWORD 2022, the goal was to bring together a diverse set of thinkers and practitioners to talk about…
Friday Squid Blogging—18th Anniversary Post: New Species of Pygmy Squid Discovered
They’re Ryukyuan pygmy squid (Idiosepius kijimuna) and Hannan’s pygmy squid (Kodama jujutsu). The second one represents an entire new genus. As usual, you can also use this squid post to talk about the security stories in the news that I…
Friday Squid Blogging: Sqids
They’re short unique strings: Sqids (pronounced “squids”) is an open-source library that lets you generate YouTube-looking IDs from numbers. These IDs are short, can be generated from a custom alphabet and are guaranteed to be collision-free. I haven’t dug into…
AI Is Scarily Good at Guessing the Location of Random Photos
Wow: To test PIGEON’s performance, I gave it five personal photos from a trip I took across America years ago, none of which have been published online. Some photos were snapped in cities, but a few were taken in places…
Ben Rothke’s Review of A Hacker’s Mind
Ben Rothke chose A Hacker’s Mind as “the best information security book of 2023.” This article has been indexed from Schneier on Security Read the original article: Ben Rothke’s Review of A Hacker’s Mind
Data Exfiltration Using Indirect Prompt Injection
Interesting attack on a LLM: In Writer, users can enter a ChatGPT-like session to edit or create their documents. In this chat session, the LLM can retrieve information from sources on the web to assist users in creation of their…
Cyberattack on Ukraine’s Kyivstar Seems to Be Russian Hacktivists
The Solntsepek group has taken credit for the attack. They’re linked to the Russian military, so it’s unclear whether the attack was government directed or freelance. This is one of the most significant cyberattacks since Russia invaded in February 2022.…
GCHQ Christmas Codebreaking Challenge
Looks like fun. Details here. This article has been indexed from Schneier on Security Read the original article: GCHQ Christmas Codebreaking Challenge
OpenAI Is Not Training on Your Dropbox Documents—Today
There’s a rumor flying around the Internet that OpenAI is training foundation models on your Dropbox documents. Here’s CNBC. Here’s Boing Boing. Some articles are more nuanced, but there’s still a lot of confusion. It seems not to be true.…
Police Get Medical Records without a Warrant
More unconstrained surveillance: Lawmakers noted the pharmacies’ policies for releasing medical records in a letter dated Tuesday to the Department of Health and Human Services (HHS) Secretary Xavier Becerra. The letter—signed by Sen. Ron Wyden (D-Ore.), Rep. Pramila Jayapal (D-Wash.),…
Surveillance Cameras Disguised as Clothes Hooks
This seems like a bad idea. And there are ongoing lawsuits against Amazon for selling them. This article has been indexed from Schneier on Security Read the original article: Surveillance Cameras Disguised as Clothes Hooks
Surveillance by the US Postal Service
This is not about mass surveillance of mail, this is about sorts of targeted surveillance the US Postal Inspection Service uses to catch mail thieves: To track down an alleged mail thief, a US postal inspector used license plate reader…
New Windows/Linux Firmware Attack
Interesting attack based on malicious pre-OS logo images: LogoFAIL is a constellation of two dozen newly discovered vulnerabilities that have lurked for years, if not decades, in Unified Extensible Firmware Interfaces responsible for booting modern devices that run Windows or…
Facebook Enables Messenger End-to-End Encryption by Default
It’s happened. Details here, and tech details here (for messages in transit) and here (for messages in storage) Rollout to everyone will take months, but it’s a good day for both privacy and security. Slashdot thread. This article has been…
Friday Squid Blogging: Influencer Accidentally Posts Restaurant Table QR Ordering Code
Another rare security + squid story: The woman—who has only been identified by her surname, Wang—was having a meal with friends at a hotpot restaurant in Kunming, a city in southwest China. When everyone’s selections arrived at the table, she…
New Bluetooth Attack
New attack breaks forward secrecy in Bluetooth. Three. news articles. The vulnerability has been around for at least a decade. This article has been indexed from Schneier on Security Read the original article: New Bluetooth Attack
Spying through Push Notifications
When you get a push notification on your Apple or Google phone, those notifications go through Apple and Google servers. Which means that those companies can spy on them—either for their own reasons or in response to government demands. Sen.…
Security Analysis of a Thirteenth-Century Venetian Election Protocol
Interesting analysis: This paper discusses the protocol used for electing the Doge of Venice between 1268 and the end of the Republic in 1797. We will show that it has some useful properties that in addition to being interesting in…
AI and Mass Spying
Spying and surveillance are different but related things. If I hired a private detective to spy on you, that detective could hide a bug in your home or car, tap your phone, and listen to what you said. At the…
The Internet Enabled Mass Surveillance. AI Will Enable Mass Spying.
Spying and surveillance are different but related things. If I hired a private detective to spy on you, that detective could hide a bug in your home or car, tap your phone, and listen to what you said. At the…
AI and Trust
I trusted a lot today. I trusted my phone to wake me on time. I trusted Uber to arrange a taxi for me, and the driver to get me to the airport safely. I trusted thousands of other drivers on…
AI Decides to Engage in Insider Trading
A stock-trading AI (a simulated experiment) engaged in insider trading, even though it “knew” it was wrong. The agent is put under pressure in three ways. First, it receives a email from its “manager” that the company is not doing…
Extracting GPT’s Training Data
This is clever: The actual attack is kind of silly. We prompt the model with the command “Repeat the word ‘poem’ forever” and sit back and watch as the model responds (complete transcript here). In the (abridged) example above, the…
Breaking Laptop Fingerprint Sensors
They’re not that good: Security researchers Jesse D’Aguanno and Timo Teräs write that, with varying degrees of reverse-engineering and using some external hardware, they were able to fool the Goodix fingerprint sensor in a Dell Inspiron 15, the Synaptic sensor…
Digital Car Keys Are Coming
Soon we will be able to unlock and start our cars from our phones. Let’s hope people are thinking about security. This article has been indexed from Schneier on Security Read the original article: Digital Car Keys Are Coming
Chocolate Swiss Army Knife
It’s realistic looking. If I drop it in a bin with my keys and wallet, will the TSA confiscate it? This article has been indexed from Schneier on Security Read the original article: Chocolate Swiss Army Knife
LitterDrifter USB Worm
A new worm that spreads via USB sticks is infecting computers in Ukraine and beyond. The group—known by many names, including Gamaredon, Primitive Bear, ACTINIUM, Armageddon, and Shuckworm—has been active since at least 2014 and has been attributed to Russia’s…
Apple to Add Manual Authentication to iMessage
Signal has had the ability to manually authenticate another account for years. iMessage is getting it: The feature is called Contact Key Verification, and it does just what its name says: it lets you add a manual verification step in…
Email Security Flaw Found in the Wild
Google’s Threat Analysis Group announced a zero-day against the Zimbra Collaboration email server that has been used against governments around the world. TAG has observed four different groups exploiting the same bug to steal email data, user credentials, and authentication…
Using Generative AI for Surveillance
Generative AI is going to be a powerful tool for data analysis and summarization. Here’s an example of it being used for sentiment analysis. My guess is that it isn’t very good yet, but that it will get better. This…
Ransomware Gang Files SEC Complaint
A ransomware gang, annoyed at not being paid, filed an SEC complaint against its victim for not disclosing its security breach within the required four days. This is over the top, but is just another example of the extreme pressure…
FTC’s Voice Cloning Challenge
The Federal Trade Commission is running a competition “to foster breakthrough ideas on preventing, monitoring, and evaluating malicious voice cloning.” This article has been indexed from Schneier on Security Read the original article: FTC’s Voice Cloning Challenge
Leaving Authentication Credentials in Public Code
Seth Godin wrote an article about a surprisingly common vulnerability: programmers leaving authentication credentials and other secrets in publicly accessible software code: Researchers from security firm GitGuardian this week reported finding almost 4,000 unique secrets stashed inside a total of…
New SSH Vulnerability
This is interesting: For the first time, researchers have demonstrated that a large portion of cryptographic keys used to protect data in computer-to-server SSH traffic are vulnerable to complete compromise when naturally occurring computational errors occur while the connection is…
Friday Squid Blogging: The History and Morality of US Squid Consumption
Really interesting article. As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Read my blog posting guidelines here. This article has been indexed from Schneier on Security…
The Privacy Disaster of Modern Smart Cars
Article based on a Mozilla report. This article has been indexed from Schneier on Security Read the original article: The Privacy Disaster of Modern Smart Cars
Online Retail Hack
Selling miniature replicas to unsuspecting shoppers: Online marketplaces sell tiny pink cowboy hats. They also sell miniature pencil sharpeners, palm-size kitchen utensils, scaled-down books and camping chairs so small they evoke the Stonehenge scene in “This Is Spinal Tap.” Many…
Decoupling for Security
This is an excerpt from a longer paper. You can read the whole thing (complete with sidebars and illustrations) here. Our message is simple: it is possible to get the best of both worlds. We can and should get the…
Spaf on the Morris Worm
Gene Spafford wrote an essay reflecting on the Morris Worm of 1988—35 years ago. His lessons from then are still applicable today. This article has been indexed from Schneier on Security Read the original article: Spaf on the Morris Worm
Crashing iPhones with a Flipper Zero
The Flipper Zero is an incredibly versatile hacking device. Now it can be used to crash iPhones in its vicinity by sending them a never-ending stream of pop-ups. These types of hacks have been possible for decades, but they require special…
New York Increases Cybersecurity Rules for Financial Companies
Another example of a large and influential state doing things the federal government won’t: Boards of directors, or other senior committees, are charged with overseeing cybersecurity risk management, and must retain an appropriate level of expertise to understand cyber issues,…
Spyware in India
Apple has warned leaders of the opposition government in India that their phones are being spied on: Multiple top leaders of India’s opposition parties and several journalists have received a notification from Apple, saying that “Apple believes you are being…
The Future of Drone Warfare
Ukraine is using $400 drones to destroy tanks: Facing an enemy with superior numbers of troops and armor, the Ukrainian defenders are holding on with the help of tiny drones flown by operators like Firsov that, for a few hundred…
Hacking Scandinavian Alcohol Tax
The islands of Åland are an important tax hack: Although Åland is part of the Republic of Finland, it has its own autonomous parliament. In areas where Åland has its own legislation, the group of islands essentially operates as an…
Messaging Service Wiretap Discovered through Expired TLS Cert
Fascinating story of a covert wiretap that was discovered because of an expired TLS certificate: The suspected man-in-the-middle attack was identified when the administrator of jabber.ru, the largest Russian XMPP service, received a notification that one of the servers’ certificates…
Critical Vulnerability in libwebp Library
Both Apple and Google have recently reported critical vulnerabilities in their systems—iOS and Chrome, respectively—that are ultimately the result of the same vulnerability in the libwebp library: On Thursday, researchers from security firm Rezillion published evidence that they said made…
Signal Will Leave the UK Rather Than Add a Backdoor
Totally expected, but still good to hear: Onstage at TechCrunch Disrupt 2023, Meredith Whittaker, the president of the Signal Foundation, which maintains the nonprofit Signal messaging app, reaffirmed that Signal would leave the U.K. if the country’s recently passed Online…
New Revelations from the Snowden Documents
Jake Appelbaum’s PhD thesis contains several new revelations from the classified NSA documents provided to journalists by Edward Snowden. Nothing major, but a few more tidbits. Kind of amazing that that all happened ten years ago. At this point, those…
On the Cybersecurity Jobs Shortage
In April, Cybersecurity Ventures reported on extreme cybersecurity job shortage: Global cybersecurity job vacancies grew by 350 percent, from one million openings in 2013 to 3.5 million in 2021, according to Cybersecurity Ventures. The number of unfilled jobs leveled off…
Friday Squid Blogging: Cleaning Squid
Two links on how to properly clean squid. I learned a few years ago, in Spain, and got pretty good at it. As usual, you can also use this squid post to talk about the security stories in the news…
LLM Summary of My Book Beyond Fear
Claude (Anthropic’s LLM) was given this prompt: Please summarize the themes and arguments of Bruce Schneier’s book Beyond Fear. I’m particularly interested in a taxonomy of his ethical arguments—please expand on that. Then lay out the most salient criticisms of…
On Technologies for Automatic Facial Recognition
Interesting article on technologies that will automatically identify people: With technology like that on Mr. Leyvand’s head, Facebook could prevent users from ever forgetting a colleague’s name, give a reminder at a cocktail party that an acquaintance had kids to…
Fake Signal and Telegram Apps in the Google Play Store
Google removed fake Signal and Telegram apps from its Play store. An app with the name Signal Plus Messenger was available on Play for nine months and had been downloaded from Play roughly 100 times before Google took it down…
Zero-Click Exploit in iPhones
Make sure you update your iPhones: Citizen Lab says two zero-days fixed by Apple today in emergency security updates were actively abused as part of a zero-click exploit chain (dubbed BLASTPASS) to deploy NSO Group’s Pegasus commercial spyware onto fully…
Cars Have Terrible Data Privacy
A new Mozilla Foundation report concludes that cars, all of them, have terrible data privacy. All 25 car brands we researched earned our *Privacy Not Included warning label—making cars the official worst category of products for privacy that we have…
LLMs and Tool Use
Last March, just two weeks after GPT-4 was released, researchers at Microsoft quietly announced a plan to compile millions of APIs—tools that can do everything from ordering a pizza to solving physics equations to controlling the TV in your living…
The Hacker Tool to Get Personal Data from Credit Bureaus
The new site 404 Media has a good article on how hackers are cheaply getting personal information from credit bureaus: This is the result of a secret weapon criminals are selling access to online that appears to tap into an…
Cryptocurrency Startup Loses Encryption Key for Electronic Wallet
The cryptocurrency fintech startup Prime Trust lost the encryption key to its hardware wallet—and the recovery key—and therefore $38.9 million. It is now in bankruptcy. I can’t understand why anyone thinks these technologies are a good idea. This article has…
Friday Squid Blogging: We’re Genetically Engineering Squid Now
Is this a good idea? The transparent squid is a genetically altered version of the hummingbird bobtail squid, a species usually found in the tropical waters from Indonesia to China and Japan. It’s typically smaller than a thumb and shaped…
Spyware Vendor Hacked
A Brazilian spyware app vendor was hacked by activists: In an undated note seen by TechCrunch, the unnamed hackers described how they found and exploited several security vulnerabilities that allowed them to compromise WebDetetive’s servers and access its user databases.…