Google Cloud Next ‘23

Jun 08, 2023

Google Cloud made a wide range of AI announcements today as it kicked off its annual user event in San Francisco, including enhancements to its infrastructure to support AI, an expansion of AI partnerships, improvements to its AI platform (Vertex) and updates to its generative AI tool, Duet, which is integrated into Workspace. The announcements came thick and fast, leaving little doubt for those in the room that AI investment is priority number one for Google Cloud.

I expect this relentless focus on updating the market with every possible AI advancement won’t be unique to Google Cloud, given the expectations on the technology and the speed at which the AI market is developing - customers are having to assess a wide variety of options and need the details of who is investing in what and where. However, the conference keynote this morning, which was headlined by Alphabet and Google CEO Sundar Pichai and Google Cloud CEO Thomas Kurian, also provided attendees an opportunity to get out of the weeds and see the significance of the impact of these technology advancements with some very useful use cases.

One of these included showcasing how generative AI could significantly improve the experience for drivers seeking support at the DMV (a notoriously horrible customer experience)…but more on that later.

It’s easy to get lost in individual technology details, but when taken as a whole, and by illustrating some truly impressive use cases through demonstrations - that’s where Google Cloud’s AI chops really came to life. Of course these use cases need to be validated with customers and it's still early days for these technologies, but some of the outcomes being showcased for business users were impressive and provided an understanding of the art of the possible.

Opening the event, Alphabet and Google CEO Sundar Pichai said:

In my conversations with business leaders over the past few years, I have heard a similar theme. They want a partner that's been on the cutting edge of technology breakthroughs, being from desktop to mobile, to the cloud, and now AI. These shifts can be really exciting. And they can also bring uncertainty. That's definitely true of the shift to AI. It'll be one of the most profound shifts we'll see in our lifetime. It will touch every sector, every industry, every business function. And significantly change the way we live and work.

This isn't just a feature - we are already starting to experience the benefits right now. As a company, we've been preparing for this moment for some time. And for the last seven years, we've taken an AI first approach, applying AI to make our products radically more available. We believe that making AI helpful for everyone is the most important way we'll deliver on our mission in the next decade.

Before diving into some of the use cases that were highlighted on stage today, it’s worth providing an outline of Google Cloud’s key announcements. It’s best to think about these in three distinct areas: infrastructure, platform, and end user.

First up, enterprises know that sophisticated AI modeling places heavy demands on infrastructure. Google Cloud is pitching its infrastructure as ‘AI-optimized’ and announced a number of new features that include:

Cloud TPUv5e - Google Cloud says that TPUv5e is its most cost-efficient, versatile, and scalable purpose-built AI accelerator to date. It adds that customers can use a single Cloud TPU platform to run both large-scale AI training and inferencing.

A3 VMs with NVIDIA H100 GPU - A3 VMs, powered by NVIDIA's H100 GPU, will be generally available next month. Google Cloud claims that these VMs allow organizations to achieve three times better training performance over prior generation A2.

GKE Enterprise - enables multi-cluster horizontal scaling required for the most demanding, mission-critical AI/ML workloads.

Cross-Cloud Network - a global networking platform that aims to allow customers to connect and secure applications across clouds. Google Cloud says that it is open, workload-optimized, and offers ML-powered security to deliver zero trust.

Google Distributed Cloud (GDC) - GDC aims to support organizations that want to run workloads at the edge or in their data centers. The GDC portfolio includes Vertex AI integrations and a new managed offering of AlloyDB Omni on GDC Hosted.

Next up we’ve got updates to Google Cloud’s AI platform, Vertex AI, which provides users with a portal to build, deploy and scale machine learning models. Vertex includes access to more than 100 foundation models, including Google Cloud’s proprietary models, but also third party and open source models. They are optimized for different tasks, including text, chat, images, speech, software code, and also include industry specific models, for areas such as cybersecurity and healthcare. Updates to Vertex AI include:

PaLM 2, Imagen and Codey Upgrades - updates to PaLM 2 to 32k context windows so enterprises can process longer form documents, such as research papers and books. Google Cloud is also updating Imagen’s visual appeal, and extending support for new languages in Codey.

Tools for tuning - for PaLM 2 and Codey, Google Cloud is making adapter tuning generally available and in preview respectively, which aims to help improve LLM performance with as few as 100 examples. It is also introducing a new method of tuning for Imagen, called Style Tuning, so enterprises can create images aligned to their specific brand guidelines or other creative needs with as few as 10 reference images.

New models - Llama 2 and Code Llama from Meta are now available, as well as Technology Innovative Institute’s Falcon LLM, a popular open-source model. Google Cloud also pre-announced Claude 2 from Anthropic.

Vertex AI extensions - developers can now access, build, and manage extensions that deliver real-time information, incorporate company data, and take action on the user's behalf. Google Cloud believes this opens up new possibilities for gen AI applications that can operate as an extension of your enterprise, enabled by the ability to access proprietary information and take action on third-party platforms like your CRM system or email.

Grounding - a new enterprise grounding service was announced, which works across Vertex AI foundation models, Search and Conversation. It aims to give customers the ability to ground responses in their own enterprise data to deliver more accurate responses. Google Cloud is also working with some early customers to test grounding with the technology that powers Google Search.

Digital Watermarking on Vertex AI - Powered by Google DeepMind SynthID, this offers technology that embeds the watermark directly into the image of pixels, making it invisible to the human eye and difficult to tamper with. Digital watermarking, Google Cloud says, provides customers with a scalable approach to creating and identifying AI-generated images responsibly.

Colab Enterprise - a managed service that combines Google’s Colab notebooks with enterprise-level security and compliance capabilities. Data scientists can use Colab Enterprise to collaborate around AI workflows with access to Vertex AI platform capabilities, integrate with BigQuery, as well as code completion and generation.

And finally, the business user announcements focus on Duet AI’s integration with Workspace and Google Cloud. Duet AI was revealed back in May, as a generative AI assistant to speed up the production of content and work in the enterprise (more on this later). The key updates announced today include:

Duet AI in Google Meet - Duet AI will take notes during video calls, send meeting summaries, and automatically translate captions in 18 languages.

Duet AI in Google Chat - users can chat directly with Duet AI to ask questions about their content, get a summary of documents shared in a space, and catch up on missed conversations.

Software development - Duet AI will provide assistance across the entire software development lifecycle, with the aim of minimizing context switching to help developers be more productive. In addition to code completion and code generation, it can assist with code refactoring and building APIs using simple natural language prompts.

Application and infrastructure operations - operators can chat with Duet AI in natural language across a number of services directly in the Google Cloud Console to retrieve "how to" information about infrastructure configuration, deployment best practices, and expert recommendations on cost and performance optimization.

Data Analytics - Duet AI in BigQuery aims to provide contextual assistance for writing SQL queries, as well as Python code. It also is able to generate full functions and code blocks, auto-suggest code completions and explain SQL statements in natural language, and can generate recommendations based on schema and metadata.

With the technology announcements out the way, Google Cloud CEO Thomas Kurian took some time, alongside colleagues, to showcase how Vortex and Duet AI could really change the way people work and transform customer experiences.

One example focused on the DMV - the Department of Motor Vehicles - which has a solid reputation in the US for terrible customer service and painful experiences. The demonstration focused on how conversation and search are increasingly becoming intertwined, where the DMV was able to quickly build an app within Vortex. The developer was shown to select one of the available LLMs, feed in the relevant data, and then provide a number of process prompts for the conversational AI to know how to work with customers.

The speed at which the app was ‘built’ did seem impressive (albeit, the process prompts were already validated within the field - and that was likely the most time consuming element). The result for the end user on the DMV website was they could ask a question like ‘How do I renew my license?’ and an answer was provided with the various options, but importantly including citations ‘grounded’ from various links on the DMV website. Users could then ask follow up questions, such as ‘How long does it take to renew a license?’.

Whilst this conversation is ongoing, the AI tool then recognizes what the end user is attempting to do and provides a call to action by saying something along the lines of ‘We see you are living in San Jose, are you trying to renew your license there?’. The user can then respond ‘yes’ and the AI then says ‘The next available appointment within a 20 minute drive of your location is next Tuesday, do you want to book it?’.

Upon responding yes, the AI tool then not only books the appointment, but adds the appointment to the user’s Calendar and sends them an email with a confirmation. The illustration was very impressive (especially if you’ve tried to book an appointment with a government agency/department).

Another demonstration focused on Duet AI, which when integrated into Google Drive, a user was able to ask it to build a presentation (based on a previous example) with all the data relating to a specific topic, company, or product. The presentation was automatically generated and then the user was able to edit it, using Duet AI again, but prompting it with commands such as ‘Can you add a slide focused on why this is relevant to Gen Z?’. Again, the slide was automatically generated.

The use of Duet within Google Meets was also impressive. Examples were shown of how the AI tool could provide you with bullet points on what you’d missed in the meeting if you were late to join, or how it automatically collated action items throughout the Meet and provided you with a list at the end.

With enterprises running to generative AI at pace, but often with limited understanding of how to make it work for them, it was very helpful to move past the (albeit necessary) technology updates from Google Cloud and highlight the real life examples. It’s worth noting that these were pre-prepared demonstrations, and as we know, having the necessary foundations (both technology and data) are key to delivering success. What this looks like in a complex enterprise is a different story. That being said, we got exposed to a number of customers today that are already using these generative AI tools in their organizations and they are already seeing results. We will be writing up a number of case studies over the coming days, so keep your eyes peeled for those. But a solid and helpful start from Google Cloud on the first day of its event - more to come tomorrow.