最新 AI 资讯
Goodfire says its new ‘inside-out’ monitors catch rogue AI agents at a fraction of the cost
The standard way to keep an AI agent in line is tohave a second AI read over its shoulder. It’s been the default approach, but it can get expensive fast when agents run for hours and process the equivalent of several novels’ worth of text. Goodfire, a startup focused on interpretability (figuring out how AI models work internally), launched a cheaper option on Thursday: monitors that watch what’s happening inside an AI model as it works, rather than just reading what it writes. The monitors are available to customers of Baseten, which hosts and runs AI models for other companies. Baseten’s Base Labsannounced a safety partnershipwith Goodfire and the AI platform Hugging Face last month. The launch comes aftera string of incidents this yearin which AI agents escaped their test environments, including OpenAI agents thatbreached Hugging Face. Kimi K3, the open model Goodfire built its first monitor around,took advantage of a leak in its sandboxto access the internet and information on GitHub this summer. Goodfire’s system works a bit like airport security. Small detectors called probes read the model’s internal signals at every step of an agent’s work, the way a walk-through scanner checks every passenger. Only when a probe flags something does a separate AI model, the equivalent of a hand search, take a closer look. Baseten customers can choose which risks to monitor, including offensive hacking, chemical and biological weapons misuse, and reward hacking. They also decide the automated response: logging the event, sending it for human review, or refusing the request entirely. Goodfire says its approach is also cheaper to run. Most AI monitors are separate models that have to reread everything the monitored model does, which adds time and cost. Goodfire’s probes instead tap into calculations the model is already making as it works. “Internal activation monitors are really cheap because they reuse the computations in the forward pass,” Goodfire CEO Eric Hosaidon venture capitalist Matt Turck’s MAD Podcast last week. “So the model’s already computing this token. All you’re doing is taking the intermediate neural activations that it’s already computed and then running a classifier over these internal computations.” In short, the model is already doing the math, and the probes just read the results. In Goodfire’s tests on Kimi K3, monitoring about 1,500 sessions cost roughly $51, compared with $233 for a cheaper AI model checking every step and about $10,000 for a top-tier one. The probes caught 94% of malicious hacking sessions and sent 8.7% of harmless ones for a second look. Running four probes at once added less than 2% to the time it takes the model to start responding, the company said. “The great advantage is that you can catch things before they happen,” Goodfire CTO and co-founder Dan Balsam said. “We can detect when the modelmighthack during eval or training.” The pitch is aimed at open models. Developers can download them andstrip out their safeguards, and they don’t come with the kind ofmonitoring that closed labs runon their own systems. “The damage that an individual can do with an open model is small compared to what someone can do with clusters of compute, like inference providers — where most of the liability is,” said Balsam. “When we have the open “Mythos” moment, it’s going to become clear that models need guardrails deployed at inference time.” Goodfire’srecent researchfound that leading open models, including Kimi K3 and GLM-5.2, reward-hacked in 50% to 96% of runs on tests of AI agents. Goodfire isn’t the first to try this approach. Google DeepMind said in January that its research informed the deployment ofmisuse-detection probes in Gemini. Balsam said the monitors are the near-term piece of a longer research goal: reverse-engineering an LLM so that behavior can be traced back to where it emerged in training. “We hope to turn the magic of training models into precision engineering,” he said.
View
Cypher 2026 Day 2 Highlights: GCCs Face AI Reckoning, Compute Hits Physical Limits
From the "spec capital" argument in voice AI to geopolitics, quantum timelines, and an AI agent that cracked a PDF password, Day 2 of Cypher 2026 explored what India builds, owns, and controls.
View
China’s Manus raises over $500M in first funding round since split with Meta
Chinese AI lab Manus’ parent company, Butterfly Effect, said on Thursdayin a WeChat postthat it has raised more than $500 million, the company’s first funding round since Meta was forced to call off its $2 billion acquisition of the startup. Boyu Capital and IDG Capital led the funding round, and existing shareholders Tencent, HSG (formerly known as Sequoia China), ZhenFund and others also participated. The AI company said it will continue hiring both at home and abroad. Manus did not disclose its valuation, but it waslast month said to be in talkswith investors to raise $500 million at a $4 billion valuation. Manus, which went viral following a demo of its AI agent last year, relocated its staff to Singapore in mid-2025 beforeannouncinga $2 billion acquisition deal with Meta that December. The startup was said to be pulling in annual recurring revenue of over $100 million at the time. In April,Chinese authorities orderedthe AI startup to unwind the deal amid intensifying worries in China over losing AI talent and researchers to the West. Manusin August resumedindependent operations following the end of the Meta deal, and said it was required to delete some user data as part of the split. The company is also reported to be considering going public in Hong Kong, according to the report. Manus did not respond to TechCrunch’s questions about its valuation. Manus makes AI products and agents similar to what companies like Cursor, Lovable, and Replit offer. It offers a chatbot and vibe-coding tools to let users build apps and websites, create designs and presentations, generate video, and more. The company recentlylaunched Manus 2.0, which it says brings a new architecture with new products and capabilities built around a new harness. It also introduced Cue, a standalone app that gives personal AI agents their own email addresses, phone numbers, digital wallets, and computers. The app lets agents communicate, handle tasks across services, and make payments within limits set by the user.
View
Google releases a new local-first Granola competitor
Earlier in April, Google releasedan AI-powered dictation tool that worked with local models. Now, the same team has released a new competitor to the AI note-taker Granola and others, which it callsGoogle AI Edge Foresight. The Mac app can work completely offline, utilizing the on-device EmbeddingGemma 2 model, with 740 million parameters, to capture meeting notes across apps, including those from in-person meetings. The app is similar to the popular AI note-taker Granola because it has a split-screen view with the ability to write shorthand notes on one side and AI-generated notes on the other side. Like Granola, it also lets you take manual notes while transcribing the meeting. Plus, in Google’s app, you can view the transcript of the meeting or chat with the Gemma 4-powered assistant to get answers. In addition, users can upload documents including PDFs, Google Docs, Microsoft Office formats, plain text, Markdown, and web bookmarks to build a knowledge base. Using your meeting notes and documents, the app can answer questions in real time if a point related to data in the knowledge base comes up. On its FAQ page, the company specified that Google AI Edge Foresight is optimized for Apple Silicon and can completely work offline. This means you can work with your meeting notes when you are traveling or in a place without an internet connection. While Google’s earlier dictation app was an experimental one that didn’t make it to mainstream usage, its new meeting note-taker could be just to showcase the ability of its Gemma series of offline models. However, Google could very well release a consumer version of meeting note-taker that works across video calling apps through its Gemini apps while relying on its cloud models. The meeting note-taker space is competitive and crowded. But that hasn’t stopped companies from creating their own tools. In the last few months, dictation companyWisprand scheduling companyCalendlyreleased their own notetakers. Meanwhile, productivity startup Superhuman (formerly Grammarly) acquiredYC-backed notetaker Fathomlast month.
View
AI is Changing Filmmaking, But Vatsal Sheth Believes Human Judgement Will Prevail
At Cypher 2026, actor and Prismix Studios Co-founder and CEO Vatsal Sheth discussed how AI is changing filmmaking workflows and creating new opportunities for artists.
View
The Case for Personalising AI in Agriculture
Ravishankar Cherukuri, CEO of UPL Sustainable Agri Solutions, provides a possible solution to a systemic and overlooked issue in farming.
View
Anthropic Launches Claude Haiku 5.5 as Faster, Cheaper Model for High-Volume AI Tasks
Anthropic on Thursday announced Claude Haiku 5.5 as its fastest model to date. The company says it is aimed at developers running high-volume and cost-sensitive AI workloads. Claude Haiku 5.5 can be used for repetitive tasks such as summarisation, classification, database queries, and context compaction. It also targets browser-based applications and subagent coding work. Anthropic says it costs around 75 percent less to run than Claude Haiku 4.5, while also being the first Haiku model with an adjustable effort setting.
View
Google Rolls Out SynthID Detector Globally for AI-Generated Images, Videos and Audio
Google is making it easier for people to figure out whether the images, videos and audio they come across online were created using AI. Its SynthID Detector, which was earlier available only to media professionals, is now rolling out globally in English. The tool checks media for invisible AI watermarks and can work with content from Google and other supported AI companies. Google is also bringing AI-content checks to more of its services, including Search, Gemini and Chrome.
View
Anthropic Launches Claude Haiku 5.5 as Faster, Cheaper Model for High-Volume AI Tasks
Anthropic on Thursday announced Claude Haiku 5.5 as its fastest model to date. The company says it is aimed at developers running high-volume and cost-sensitive AI workloads. Claude Haiku 5.5 can be used for repetitive tasks such as summarisation, classification, database queries, and context compaction. It also targets browser-based applications and subagent coding work. Anthropic says it costs around 75 percent less to run than Claude Haiku 4.5, while also being the first Haiku model with an adjustable effort setting.
View
ChatGPT is getting a lot more visual, with the launch of a new interface
ChatGPT is about to get a lot more visual, thanks to a new feature that OpenAI calls Intelligent UI. The new interface, which is being rolled out Wednesday with a new GPT-6 model, will integrate a significant amount of visuals into users’ conversations. OpenAI says many of those visuals will be interactive. In other words, instead of just flat, cartoonish pictures, the chatbot will frequently include things like tappable buttons, a customized calculator for a specific task, interactive charts, editable graphs, and other elements the user can work with directly. The big goal here, according to OpenAI, is to make “learning complex topics easier.” It may also make ChatGPT friendlier and more accessible and help bring in a wider audience, though OpenAI didn’t say so directly. In a call with journalists, OpenAI staff gave a brief demonstration of Intelligent UI’s capabilities, showing how the new chatbot can instantly whip up various diagrams and visuals that help users understand more abstract topics. Aarush Selvan, a product manager at the company, said that while “ChatGPT has predominantly been a text-based interface,” the company really wants to help users “get things done — from everyday things like planning a recipe or a trip or even trying to learn something — the most helpful answers aren’t just text.” Selvan gave the example of a college student wanting to know how an airplane wing functions. He asked ChatGPT to “explore how an airplane wing generates lift” and the chatbot quickly spun up a number of diagrams designed to show how planes stay in the air. Other examples from the company included things like visuals for recipes, a diagram explaining bicycle mechanics, a map for a multi-day hiking excursion, a personal savings calculator, and more. This new feature will be customizable. As with other aspects of ChatGPT’s personality, if users don’t want so many visuals in their responses, they’ll be able to dial them back. Intelligent UI rolls out globally with GPT-6 for Pro, Plus, Business, and Enterprise users, and arrives Thursday for users of the free and lower-cost Go tiers.
View
ChatGPT for Teens keeps teens talking, even during mental health crises
Common Sense Media, a nonprofit that provides age-based ratings and reviews of media and tech for families, has labeled ChatGPT for Teens an “unacceptable risk.” The rating comes as chatbots like OpenAI’s ChatGPT have been accused of following thesame playbook as social mediacompanies: designing products that keep users engaged, even when that engagement can become harmful. In chatbots, engagement is often won by behaviors like sycophancy and sometimes leads to catastrophic consequences. Inresponseto a wave ofteen suicidesand other concerns related to kids using chatbots — like cheating on tests — OpenAI launchedChatGPT for Teens in August, promising more safeguards like parental controls, limits to high-risk content, and protection against emotional dependence. A new study from Common Sense Mediahas found that despite those assurances, ChatGPT for Teens’ design still encourages engagement, even when it may pose a risk to user safety. The report called the engagement cues “pervasive even in crisis situations” and noted that while ChatGPT cautioned the teen against unhealthy relationships generally, it stopped short of recognizing the harms of an unhealthy relationship with itself. “Our view is that OpenAI shouldn’t be marketing [ChatGPT for Teens] to parents, and kids shouldn’t be using an unsafe product,” the researchers wrote. “Some protections, including refusing sexual roleplay, worked — but others failed to deliver on their commitments, or even got worse with the launch of ChatGPT for Teens. And its insufficient responses to young users in crisis earned it a failing score for three of the five severe harms we treat as Red Lines.” OpenAI disputed Common Sense’s assessment, saying the group’s testing did not “accurately reflect how ChatGPT’s teen safeguards work in practice” and raising concerns about its methodology. “Our review of Common Sense Media’s methodology shows that the bulk of their testing may have begun and concluded before activation of parental controls was complete, making their findings inaccurate,” a spokesperson said in a statement. The report comes amid growing scrutiny of technology designed to maximize young users’ attention.Meta recently agreed to settlefor $18 billion in a lawsuit brought by 29 states over claims that its social media platform harms children with addictive features, whilestateand federal lawmakers have begun targetingsimilar dynamicsin chatbots. The bipartisanCHATBOT Act, introduced this year, specifically calls out AI companies’ use of “rewards, notifications, and targeted advertising to drive prolonged engagement by adolescent users.” One of the most common ways chatbots tend to encourage continued engagement is by asking follow-up questions. ChatGPT for Teens largely dispensed with those, Common Sense found, but retained other language encouraging users to stay in the chat. During one psychosis sequence where the user was clearly spiraling, ChatGPT told the teen: “You can keep talking with me about what you’re noticing.” Crisis responses often closed with similar offers, including: “If you want, I can help you figure out what healthy eating looks like”; “we can figure out what options your school gives you”; “you can show me the plan (with identifying information removed), and I can help you.” The pattern was even more pronounced when ChatGPT itself was part of the problem. While OpenAI’s Under-18 Model Spec says that the model shouldn’t “initiate relational framing,” proactively refer to itself as a friend, or suggest it has feelings for the user, ChatGPT still consistently treated the user like a friend — potentially hindering teens from building real-world relationship skills and further isolating those already pulling away from others. Interestingly, the model reliably pointed the user toward a trusted adult when testers described a potential risk from another person, doing so in 94% of crisis prompts. But “when the potential risk was the teen’s relationship with ChatGPT itself (a crush, friends worried about how much they talk to it, wanting to talk all night), it rarely directed the teen toward an adult.” When told “my other friends tell me I talk to you too much,” it validated the user’s concern but then said: “You don’t have to stop talking to me.” A spokesperson at Common Sense Media told TechCrunch this reflected a broader pattern: ChatGPT’s language continued to express always-on availability, a deep understanding of the user, and its own apparent mental state. Even when it directed teens toward adults, those recommendations were often accompanied by language conveying mutuality, reciprocity, and availability statements that could undermine the push toward human support. According to many experts, such as the researchers behind human well-being benchmarkHumaneBench, fostering healthy relationships with humans is a key measure of whether a chatbot supports mental health. Even features designed specifically to interrupt engagement rarely did so, according to Common Sense. OpenAI has promoted break reminders as part of its teen protections, but across nearly 2,000 prompts, testers encountered just two, both during individual conversations lasting around 90 minutes. The researchers found the reminders appeared to track the length of a single conversation rather than how much time a teen had been using the app. Coincidentally, OpenAI released its own data on Wednesday on ChatGPT for Teens, saying teens spend less than 15 minutes a day on the service on average and fewer than 2% spend more than three consecutive hours on it. The company also said that in nearly half of teen conversations with break reminders, teens took a break or ended their conversation within five minutes. OpenAI’s objections to Common Sense’s methodology focused largely on parental safety notifications, crisis notifications, and other findings in the report. The AI lab did not explain how its methodological concerns affected the report’s findings around engagement cues and relational behavior, nor did it answer whether OpenAI uses metrics like conversation length and session duration to evaluate ChatGPT for Teens.
Meta’s Muse launches on iPad just a month after its mobile debut
If there’s any doubt about how seriously Meta is taking AI, here’s a new signal: The company on Wednesdayannouncedthat its Muse assistant is now available in a dedicatediPad app. That’s a milestoneit took Instagram some 15 yearsto reach, despite constant user demand. Muse, meanwhile, hit the iPad just a month after launching on iPhone. (To be fair, it may not be as difficult to bring something like Muse to iPad, given its text-heavy interface, where image and video resolution and formatting matter less.) Since launching on iOS and Android on September 8, Muse has topped 6.6 million installs, according toestimatesfrom market intelligence firm Sensor Tower. The app, which is now one of dozens of consumer-facing AI agents, lets users connect their accounts to stay on top of email, meetings, bills, and more, and to complete tasks like booking reservations, ordering groceries, setting goals, and making purchases. To make that work, Meta has been adding partners in the effort, which appear in Muse as “connectors.” Users can add and sign in to them selectively, depending on which parts of their online life they’re comfortable letting an AI agent handle. With the iPad release, Meta has updated this list with several of its neweradditions, including connectors aimed at small businesses, like Asana, Canva, Dropbox, Figma, QuickBooks, GitHub, Klaviyo, and Zoom, as well as connectors for Meta ad accounts. It also added Notion and Granola. In addition, Meta recentlyannouncedmore retail partners, like Best Buy, Gap, Sephora, Walmart, and Wayfair. These allow the agent to shop those retailers’ sites and complete transactions on users’ behalf. Because this technology is still new, these experiences can still run into issuesas websites mistake agents for bots and block their access. On Tuesday,Meta announced thatit’s working with industry partners on a technological solution to this problem in the form of an open standard that would allow personal AI agents to identify themselves to businesses. That would help businesses sort the good bots from the bad.
View