Posts for Tag: AWS

Amazon AWS Re:Invent 2023 Ushers in a New Era for AWS

AWS recently held its annual re:Invent conference, showcasing exciting new offerings that demonstrate the company's continued leadership in cloud computing and artificial intelligence. This year's event had a strong focus on how AWS is pioneering innovations in generative AI to provide real business value to customers.

CEO Adam Selipsky and VP of Data and AI Swami Sivasubramanian headlined the event, announcing breakthrough capabilities spanning hardware, software, and services that mark an inflection point for leveraging AI. AWS is committed to progressing generative AI from leading-edge technology into an essential driver of productivity and insight across industries.

Highlights from Major Announcements

Here are some of the most notable announcements that give a glimpse into the cutting-edge of what AWS is building:

  • Amazon Q - A new AI-powered assistant designed for workplace collaboration that can generate content and code to boost team productivity.  
  • AWS Graviton4 and Trainium2 Chips – The latest generation AWS processor and accelerator chips engineered to enable heavy AI workloads like training and inference.  
  • Amazon Bedrock Expansion – New options to deploy and run custom models and automate AI workflows to simplify integration.
  • Amazon SageMaker Updates – Enhanced capabilities for novices and experts alike to build, train, tune and run machine learning models faster. 
  • Amazon Connect + Amazon Q - Combining AI assistance and customer service software to help agents respond to customers more effectively.

AWS underscored its commitment towards an intelligent future with previews showcasing bleeding edge innovation. This vision crystallizes how human-AI collaboration can transform customer experiences and business outcomes when generative AI becomes an integral part of solution stacks. Re:Invent 2023 ushered in this emerging era.

As the curtain falls on AWS re:Invent 2023, the message is clear: AWS is not just keeping up with the pace of technological evolution; it is setting it. Each announcement and innovation revealed at the event is a testament to AWS's unwavering commitment to shaping a future where technology is not just a tool but a catalyst for unimaginable growth and progress. The journey of AWS re:Invent 2023 is not just about celebrating achievements; it's about envisioning and building a future that's brighter, faster, and more connected than ever before.


Exploring Amazon Bedrock Foundation Models

At CPROMPT.AI, we aim to make advanced AI accessible and valuable for everyone.  Our vision is to allow consumers, professionals, and SMBs to benefit from AI in their daily lives and work without needing expertise in artificial intelligence.

Rather than reinventing the wheel, we leverage state-of-the-art large language models via API.  This allows us to focus on our core strength - creating the best prompt engineering experience to simplify AI and maximize its value to our customers.

With CPROMPT, users can achieve remarkable results in their fields and interests through natural language-based prompt engineering without any programming.  We aim to continuously enhance our prompt engineering capabilities to empower people to use AI effectively, ethically, and safely.

We aim to drive mainstream AI adoption by abstracting away the complexity.  With CPROMPT as your AI assistant, you can spend less time learning about AI and more time using it to enhance your life and business.  The future of AI is here - simple, accessible, and for everyone.

This is why we need access to many different LLM API services to offer an array of tools to our customers.  As our engineering teams are constantly reviewing and experimenting with LLM API, so are our business analyst teams.  The latter is focused on the pricing of API services.  In this post, we will introduce the Amazon Bedrock platform and compare Open AI GPT-3.5, GPT-4, and many Amazon Bedrock Foundation Model (FM) API service costs to help us offer the best LLMs to our customers.

Introduction to Amazon Bedrock

Amazon Bedrock is a new, fully managed service that makes building and deploying advanced generative AI applications easy.  With Bedrock, developers can access leading foundation models from top AI companies like Anthropic, Meta, and Cohere through a single API.  This allows you to experiment with different models and choose the best one for your needs with minimal effort. 

Bedrock's critical capability is customizing foundation models using your data.  Using techniques like fine-tuning and retrieval augmented generation, you can tailor the models to your specific use cases and generate high-quality outputs personalized to your business.  Bedrock also enables the creation of AI agents that automate complex tasks end-to-end, from booking travel to processing insurance claims.  It handles the underlying infrastructure and operations so that you can focus on the tasks rather than managing the servers.

Integration with other AWS services is seamless, allowing you to deploy generative AI where you need it securely.  And with the serverless consumption model, you only pay for what you use.  Overall, Amazon Bedrock simplifies and accelerates the development of generative AI applications.  By providing easy access to leading models, customization capabilities managed by AI agents, and secure integration with AWS, Bedrock makes it faster for developers to build the next generation of AI-powered solutions.

Bedrock Foundational Models

Amazon Bedrock provides access to various high-quality foundation models from Amazon and other leading AI companies.  Developers can leverage text-based models such as Titan from Anthropic, Jurassic from AI21 Labs, and Cohere, embedding models like Meta's Command and image models including Stable Diffusion XL from Stability AI. 

With this extensive lineup of capable foundation models supporting different modalities like text, embeddings, and images, you can quickly experiment to determine the optimal model for your application.  The interactive playground makes it easy to try models like Claude, Llama, and others to evaluate which performs best for your specific use case.

By bringing together many of the most advanced foundation models in one place, Amazon Bedrock simplifies the process of testing and identifying the ideal model to meet your needs.  Whether you need sophisticated text generation, semantic search, creative image synthesis, or another capability, you can readily access leading models and compare them side-by-side.

The access to the models is not automatic.  You must submit a request to model providers and wait for them to enable the model access.  Here is a figure showing we have requested Claude's access.

Luckily, we were granted access as shown below:

What are Key Benefits of Amazon Bedrock?

There are four reasons for us to care about the Amazon Bedrock platform to access these widely available LLMs. 

Access to Closed Source LLMs via API

The closed-source ones, such as Claude, have a very restricted API release for startups like us.  So, even after repeated attempts to get them to open their API to us, we failed.  However, while we were trying, they were getting investments from Amazon, and of course, their LLM now shows up as an FM model available for access through Amazon Bedrock.  So accessing closed-source LLM through Amazon Bedrock will be a great way to access the API for our CPROMPT users.  And this is true for any small startup in the AI application space. 

Running Open Source LLM without 24/7 Infrastructure Cost

Running open-source LLM such as Llama is very tempting.  We have run our own Llama instances for testing, and the issue is maintaining your cloud infrastructure, for Llama is a huge cost center for small self-funded startups like ours.  This is why it makes sense to pay for API services for usage rather than paying for keeping the servers running with expensive Tensor or GPU hardware on a 24/7 basis.  This allows us to scale up first and then decide on dedicated private instances for our use, depending on the needs of our users.

As of this writing, Llama 2 models from Meta were still unavailable and listed as coming soon.  These open-source fine-tuned models leverage publicly available instruction datasets and over 1 million human annotations, and Llama 2 models are trained on two trillion tokens.  One of the Llama 2 fine-tuned models has over 1000 hours of red-teaming and annotation effort to ensure safety with model performance.  So, we are looking forward to having access to Llama 2 through the Amazon bedrock platform very soon and will write a post about it when it is available.

Cutting Development Time Using a Single API

Amazon Bedrock provides a single API to access and interact with all the available foundation models, regardless of the provider.  This unified API abstracts away the underlying complexity of different model architectures.

With minimal code changes, developers can switch between state-of-the-art models like Claude, Jurassic-2, Stable Diffusion XL, and more.  The API seamlessly handles inference for each model, allowing you to experiment and determine the optimal one for your needs.

As new model versions and innovations become available, you can easily leverage them through the same simple API without extensive reworking.  This unified access accelerates experimentation across a diverse model landscape.

By decoupling models from the interface, Amazon Bedrock enables flexible exploration and future-proof adoption of cutting-edge advances in foundation models.  The single API reduces friction in evaluating, comparing, and selecting the ideal model for your applications.

Data Security and Private Data Embeddings

Amazon Bedrock provides enterprise-grade capabilities to meet your security and privacy requirements when building AI applications.  Your proprietary data remains private and is not used to improve third-party foundation models.  Bedrock encrypts your content both in transit and at rest.  For additional control, you can encrypt data with your keys. 

With PrivateLink support, you can securely connect Bedrock to your Amazon VPC without internet exposure.  This enables building secure solutions meeting compliance needs.  Bedrock has achieved formal compliance with regulations, including HIPAA eligibility and GDPR.  The service provides concrete capabilities to satisfy rigorous security and privacy standards.

Whether dealing with sensitive customer information, healthcare data, or other regulated content, you can rely on Amazon Bedrock's defense-in-depth approach.  Focus on creating impactful AI applications while Bedrock handles security, privacy, and compliance requirements.

Anthropic Claude LLMs on Amazon Bedrock 

Claude represents a breakthrough in reliable, interpretable, and steerable AI systems.  Developed using cutting-edge techniques like Constitutional AI and harmlessness training, Claude truly excels at thoughtful dialogue, content creation, complex reasoning, creativity, and coding.  With an expansive extended context window of 100K, equivalent to around 75,000 words, Claude can securely process extensive amounts of information - from documents and reports to emails, chat transcripts, and more.  Claude's advanced natural language processing capabilities enable efficient handling of any text-based data you need to analyze.  Amazon bedrock has two Claude models available.

Claude Instant

Claude Instant 1.2, a scaled-down version of its Claude 2 foundation model.  Claude Instant provides an affordable way for businesses to leverage Anthropic's natural language AI capabilities.

Designed for high throughput use cases that require lower latency, Claude Instant can handle tasks like dialogue, text analysis, summarization, and document comprehension.  It charges lower per-token pricing, starting at $1.63 per 1000 tokens.

With a 100,000-token context window, Claude Instant can process approximately 75,000 words per prompt.  The model is optimized for applications needing capable generative AI at reduced cost.

Claude 2

For more advanced use cases needing complex reasoning, Anthropic recommends its enterprise-grade Claude 2 model, which starts at $11.02 per 1000 tokens.  Claude Instant makes Anthropic's technology accessible to various applications and users.

Who is Using Amazon Bedrock?

Companies like LexisNexis, Lonely Planet, and Ricoh USA are already leveraging Claude 2 on Amazon Bedrock to integrate advanced generative AI capabilities into diverse services.  LexisNexis is powering new legal AI solutions, Lonely Planet is enhancing travel planning with decades of content, and Ricoh is infusing operations with secure, high-quality data generation.  These leading organizations chose Claude 2 and Amazon Bedrock for the reliable performance, robust security, and enterprise-grade support needed to build impactful real-world AI applications across industries.  The rapid adoption exemplifies how Amazon Bedrock enables businesses to quickly capitalize on state-of-the-art foundation models like Claude 2 and drive innovation.

How Amazon Bedrock Pricing Works?

There are two different pricing schemas for Amazon bedrock.  They are discussed below.

On-demand pricing

An application developer makes the following API calls to Amazon Bedrock: A request to Anthropic's Claude model to summarize an input of 11K tokens of input text to an output of 4K tokens.

Total cost incurred is = 11K tokens/1000 x $0.01102 + 4K tokens/1000 x $0.03268 = $0.25

Provision throughput pricing

An application developer buys one model unit of Anthropic Claude Instant with a 1-month commitment for their text summarization use case.

Total monthly cost incurred is = 1 model unit X $39.60X 24 hours X 31 days = $29,462.40

Comparing Pricing with Open AI API Fees

OpenAI offers complex pricing for API-based access.  Since it provides GPT-3.5 Turbo and GPT-4 access, the pricing details for both models with context windows range from 4K to 32K.  Please note that GPT-4 32K is not available to us yet.

Here is the pricing for GPT-4 models with a context window ranging from 8K to 32K.

CPROMPT.AI offers access to GPT-3.5 Turbo 4K, 16K, and GPT-4 8K models for prompt designers.

Final Thoughts and Recommendations

Amazon Bedrock provides the comprehensive capabilities to leverage generative AI and transform your business entirely.  With Bedrock, you can easily experiment with leading foundation models, tailor them to your use cases, and build AI-powered solutions without managing infrastructure.  Fine-tune models on your data for greater accuracy and create managed AI agents to automate complex workflows from end to end.  Integrate these cutting-edge capabilities securely into your AWS environment and applications through a serverless, pay-as-you-go model.

Monitor usage through CloudWatch, maintain governance with CloudTrail, and comply with regulations, including GDPR and HIPAA.  With this combination of ready access to state-of-the-art generative AI and enterprise-grade tools for oversight, Bedrock enables you to deliver high-value AI solutions rapidly.  Unlock generative AI's immense potential and drive innovation with Amazon Bedrock.

Since Amazon Bedrock is a highly brand new platform for generative artificial intelligence access via Amazon Web Services (AWS), it can be a bit daunting for anyone who has not used AWS before. If you want to learn more about the AWS Bedrock platform, consider this book, co-authored by Antje Barth, Principal Developer Advocate at Amazon Web Services, focused on AI/ML.  Based in San Francisco, she engages global developers and data scientists in applying AWS for machine learning.  Antje co-authored "Data Science on AWS" and co-founded the worldwide "Data Science on AWS" Meetup.  A sought-after speaker at AI conferences, she helps developers and companies build and deploy cloud-based AI solutions.  Antje is passionate about big data, containers, and Kubernetes for ML.  She previously worked in technical evangelism at MapR and Cisco, advising on AI systems.