Re:Cap - D55 at Re:Invent 2023

Last week, the D55 team attended AWS re:Invent, the annual AWS conference that attracts tens of thousands of visitors eager to learn and network. Traditionally, re:Invent serves as a platform for launching numerous new features across AWS's extensive range of services. Below are some highlights we found interesting.

Step Functions

AWS Step Functions, a favourite service at D55, facilitates orchestrating actions across multiple services and handling error states with a defined process. A significant new feature is the addition of HTTP Endpoints, enabling steps in the Step Functions to make HTTP requests to non-AWS services, including customer-owned microservices or third-party APIs like Stripe. There's also a Native Bedrock Integration, ideal for applications wanting to incorporate Generative AI directly into workflows. These features reduce the need for custom code and its subsequent support.

Another major update to Step Functions is the introduction of Test States, simplifying the testing of individual states in a workflow. This enhancement can accelerate development and improve the reliability of Step Functions.

Observability

Observability, crucial for all applications, especially those distributed in a cloud-native manner, includes metrics, logs, dashboards, and alarms/notifications. Enhancements in this area include a new Patterns view for visualising recurring patterns in log queries, a compare mode for identifying changes over time, and newly released anomaly detection that assesses incoming logs against historical baselines. Together, these advancements aid in quickly identifying issues and maintaining healthy production applications.

Security

Amazon Inspector is a service from AWS which D55 makes use of for automated vulnerability management. It will continually scan workloads and look for known software vulnerabilities, allowing teams to react quickly when new vulnerabilities are discovered. Some new features were added to Inspector, including Lambda code scanning with generative AI. Lambda functions are a common way to run code in our customer environments, and the ability to not only scan for vulnerabilities, but also to suggest patches can help resolve issues quickly.

Another small yet impactful feature from AWS is the introduction of unused access finding in IAM Access Analyser. One common issue in AWS accounts is IAM policies that are too permissive, therefore allowing users and systems to access more resources than they should have access to. Having better tools to find these unused permissions means that permissions can be tightened up, increasing security, and reducing attack vectors for bad actors.

Database and Storage

Databases are a key part of all applications, and for those that use relational databases Amazon Aurora is an important service. It offers very high performance, reliability, and availability. This was extended further with the announcement of Amazon Aurora Limitless Database. This enables users to scale database clusters, to handle millions of write transactions per second and manage petabytes of data. Not all companies will need to handle that much data but knowing it can scale so highly is surely a load off your mind!

For anybody making use of Redis or Memcached, Amazon ElastiCache is the AWS managed service to run these clusters with ease. The only issue with this has been the need to calculate requirements and size the nodes to be able to handle peak load. Aside from the potential of getting this wrong, and hitting bandwidth limits, this also means paying for compute when it's not needed. A new Amazon ElastiCache Serverless option has been released to assist with this. This is based on the amount of data stored and the amount of processing required, although there is a minimum cost. Overall, this could decrease costs and increase scalability.

For anybody making use of Amazon Redshift for data warehousing, the new zero-ETL integrations could be a great boon. This includes Aurora PostgreSQL, RDS MySQL and DynamoDB. What this means is that as data is written into a DynamoDB table for example, it is available from without Amazon Redshift seamlessly! No more ETL jobs to build and manage for these specific use cases.

Amazon OpenSearch also gained some zero-ETL integrations, for Amazon S3 and Amazon DynamoDB. This could be powerful if you are using OpenSearch as the main solution for observability, as it means you can use the same dashboards and search capability, but to search across the audit logs that may be stored inexpensively within S3.

Generative AI

Of course, the main topic of re:Invent this year was AI, and especially Generative AI - a hot topic this year, especially with the viral nature of ChatGPT. The ability to generate content can be very powerful, especially when combined with organisational data. This practice is called retrieval augmented generation (RAG) and allows the power of a Large Language Model to consume private data. One example of this could be a chatbot to assist a member of staff in reviewing a loan application, the chatbot can look at the customers data and combine that with historic application data to make suggestions and speed up the retrieval and summarisation of information.

Bedrock

There were many announcements in this area, one large one was Knowledge Bases for Amazon Bedrock, which is a fully managed service to take data from a datastore in S3 and do the work of converting and storing as appropriate. This is supported by the new vector engine released for Amazon OpenSearch Serverless, as well as fully managed Agents. Agents are an exciting extension to these large foundation models, as they allow the AI model to execute tasks – such as calling APIs to connect multiple systems together.

Bedrock also saw some new fine-tuning capabilities that can specialise these generalised models to your use-case. This fine-tuning capability is useful for specialised services, for example, a company that wants to use generative AI to analyse logs and operational data and suggest resolutions to users.

To ensure safeguards are in place for these foundation models, then another announcement was Guardrails for Amazon Bedrock. This can be very important for ensuring a model doesn't return unwanted information, as well as filter out harmful content. It highlights one of the main issues with making use of these large language models – they are inherently creative and therefore not very deterministic.

Amazon Q

Amazon Q is a new service announced at re:Invent, it's actually a service I have been beta testing for a while. It's a generative AI-powered assistant that has been trained on 17 years of AWS knowledge, and you'll see the chatbot is popping up everywhere in the console and documentation. It's not really a specific service, as it's found in different forms across AWS services. The chat-bot aspect allows you to ask questions about AWS without needing to trawl documentation, and it can be quite useful, but it also has the fatal flaw of hallucinating. I have seen examples of it talking about features that don't exist, recommending the incorrect resources and sharing information that is usually not discussed outside of AWS. This chatbot also integrates into specific areas, such as if a Lambda function errors it can help troubleshoot the reason it failed.

The other guise for Amazon Q is that it can connect to business systems and use that data to answer questions, summarise reports, etc. This means it could be a way for employees to get quicker and more power from internal private data, in a secure way, and with built in fact-checking.

For customers that use Amazon Connect, the AWS service for contact centres, then a form of Amazon Q has been added there. This can suggest responses and help with training, and just generally try to speed up support.

The final area where Amazon Q has been added is within the IDE, which combines with Amazon CodeWhisperer (AI Code Suggestions) as a direct competitor to GitHub Copilot X. I have been using the beta of Copilot X for months and really like it, having a chat within the IDE that has the correct context is very powerful. More options in this area are always good, and I'll be sure to compare this new feature with Copilot. Speaking of CodeWhisperer, several new capabilities were added too that make it worth checking out if not already using another tool, like Copilot.

Finally, for those that want to try out Generative AI but don't know where to start, PartyRock was released just before re:Invent. This is a friendly fun way to build applications using generative AI, that themselves are powered by the same models. In seconds you can create a full application to generate images, come up with birthday party ideas, summarise documents, and much more.