Thursday, 13 February 2025

Introduction to Amazon Bedrock

 


Build Generative AI Applications with Foundation Models - Amazon Bedrock - AWS

Amazon Bedrock is a fully managed service that simplifies the development of generative AI applications using foundation models (FMs) from providers like Anthropic, AI21 Labs, Stability AI, and Amazon itself. 


Key features include:

  • Foundation Models: Pre-trained models that can be lightly customized using techniques like fine-tuning or Retrieval Augmented Generation (RAG) without requiring extensive ML expertise.
  • Serverless Infrastructure: No need to manage infrastructure; it provides a streamlined, API-based experience for quick deployments
  • Security and Privacy: Data is encrypted, region-specific, and not shared with model providers.


    Use Cases

    Ideal for developers looking to rapidly integrate generative AI into applications such as:

      • chatbots
      • text generation
      • image creation


    Cost Model

    Pay-as-you-go pricing based on API usage, making it cost-effective for intermittent workloads.


    Best for developers or businesses without deep ML expertise who need a fast and easy way to deploy generative AI applications.

    Friday, 7 February 2025

    AWS Secrets Manager

     


    AWS Secrets Manager allows us to:
    • Centrally store and manage credentials, API keys, and other secrets.
    • Use AWS Identity and Access Management (IAM) permissions policies to manage access to your secrets.
    • Rotate secrets on demand or on a schedule, without redeploying or disrupting active applications.
    • Integrate secrets with AWS logging, monitoring, and notification services.

    Viewing Secrets

    To list all secrets in a particular region:

    % aws secretsmanager list-secrets --region us-east-2         
    {
        "SecretList": [
            {
                "ARN": "arn:aws:secretsmanager:us-east-2:700840607999:secret:my-app/stage/my-secret-bwwria",
                "Name": "my-app/stage/my-secret ",
                "Description": "Secret for my-app in staging env",
                "LastChangedDate": "2025-01-13T12:51:21.204000+00:00",
                "LastAccessedDate": "2025-02-07T00:00:00+00:00",
                "Tags": [
                    {
                        "Key": "environment",
                        "Value": "stage"
                    },
                    {
                        "Key": "service",
                        "Value": "main-app"
                    }
                ],
                "SecretVersionsToStages": {
                    "11877f11-1999-4f37-8311-283ad04d70f1": [
                        "AWSCURRENT"
                    ],
                    "ab81397d-eb1d-4dc1-8a44-961ce45de258": [
                        "AWSPREVIOUS"
                    ]
                },
                "CreatedDate": "2022-08-17T12:55:43.194000+01:00",
                "PrimaryRegion": "us-east-2"
            },
            ...
          ]
     }


    Deleting a secret


    By default, secret is not deleted immediately but after 7 days.

    To delete a secret immediately use --force-delete-without-recovery option:

    % aws secretsmanager delete-secret --secret-id my-app/stage/my-secret --force-delete-without-recovery --region eu-west-2
    {
        "ARN": "arn:aws:secretsmanager:eu-west-2:700859607999:secret:my-app/stage/my-secret-E0yyRM",
        "Name": "my-app/stage/my-secret",
        "DeletionDate": "2025-02-07T14:54:30.386000+00:00"
    }



    Resources:


    Thursday, 6 February 2025

    Introduction to AWS S3


    S3 = Simple Storage Service

    What is Amazon S3? - Amazon Simple Storage Service

    Amazon S3 provides:
    • storage for data (objects)
      • organized as key-value structure - each object has a unique key and url
      • divided into multiple buckets; each bucket contains objects
    • web service for upload/download
    S3 doesn't know anything about files, directories, or symlinks. It's just objects in buckets. [source]

    S3 also has no concept of symbolic links created by ln -s. By default, it will turn all links into real files by making real copies. You can use aws s3 cp --no-follow-symlinks ... to ignore links. [Use S3 as major storage — GEOS-Chem on cloud]


    Buckets

    Each bucket has its own subdomain. E.g.: the one named as my-bucket would have URL:

    https://my-bucket.s3.amazonaws.com

    Objects

    • Data stored in buckets, which are logical containers
    • There is no official limit to the number of objects or amount of data that can be stored in a bucket
    • The size limit for objects is 5 TB
    • Every object has a key (object name). This is usually a name of the file. 
    Each object has its key which uniquely identifies it within a bucket. E.g. if we upload file and assign key my-root-dir/dirA/fileA1 to it, its URL will be:

    https://my-bucket.s3.amazonaws.com/my-root-dir/dirA/fileA1


    To list all objects;

    % aws s3api list-object-versions --bucket my-bucket --query='{Objects: Versions[].{Key:Key,VersionId:VersionId}}' --output=json --profile=my-profile                   
    {
        "Objects": [
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/data-1NMp6hagSAu8qORLSpyVxw.dat",
                "VersionId": "KaKyog0yM41SG._aWTuDllb9kXp67vLr"
            },
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/data-3v3MO1kOTBOl0idDYWLstA.dat",
                "VersionId": "i3PJhBpFtHrdKEtiEZPU_MFHYb2GqX0s"
            },
            ...
    }

    Versioning

    • One of the bucket features
    • Means of keeping multiple variants of an object in the same bucket
    • Used to preserve, retrieve, and restore every version of every object stored in the bucket
    • Helps recovering more easily from both unintended user actions and application failures
    • Ff Amazon S3 receives multiple write requests for the same object simultaneously, it stores all of those objects
    • Buckets can be in one of three states:
      • Unversioned (the default)
      • Versioning-enabled
      • Versioning-suspended
        • After you version-enable a bucket, it can never return to an unversioned state. But you can suspend versioning on that bucket.
    • If versioning is enabled:
      • If you overwrite an object, Amazon S3 adds a new object version in the bucket. The previous version remains in the bucket and becomes a noncurrent version. You can restore the previous version.
      • Every object, apart from its key/name, also gets its VersionId. Each version of the same object has a different VersionID. When we overwrite an object (PUT command), a new version is crated - object with the same key but a new VersionId.
      • When we delete an object, a delete marker is created and that becomes a current version. GET requests will return 404 - Not Found. But, if we pass the VersionId, GET command will return noncurrent version of the object.
      • To delete old version of the object, we need to pass VersionId in DELETE command
      • Versioning flows: How S3 Versioning works - Amazon Simple Storage Service

    Delete Markers

    • Placeholders for objects that have been deleted 
    • Created when versioning is enabled and a simple DELETE request is made 
    • The delete marker becomes the current version of the object, and the object becomes the previous version 
    • Delete markers have a key name and version ID, but they don't have data associated with them 
    • Delete markers don't retrieve anything from a GET request 
    • The only operation you can use on a delete marker is DELETE, and only the bucket owner can issue such a request 
    To list all delete markers:

    % aws s3api list-object-versions --bucket my-bucket --query '{Objects: DeleteMarkers[].{Key:Key,VersionId:VersionId}}' --output=json --profile=my-profile  

    {
        "Objects": [
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/",
                "VersionId": "hYbkDv9egrv_WE2jI4y0Lys5Bc2dbvgb"
            },
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/data-1NMp6hagSAu8qORLSpyVxw.dat",
                "VersionId": "KdLBHImFlC23EXkfq4Ic0.2x6wGJ2FxR"
            },
            ...
    }


    Bucket Lifecycle Policy


    If versioning is enabled, each time an object is updated a new version becomes the current version while previous version becomes the most recent noncurrent version. Over the time the number of object versions grows, taking more storage and driving costs up. If we want to keep only last V versions and/or we want to delete noncurrent versions after D days we can define a lifecycle policy.


    Deleting a bucket


    Before we attempt to delete a bucket we need to make sure that both all objects and all delete markers are deleted. 

    To delete all objects (current versions):

    % aws s3api delete-objects --bucket my-bucket --delete "$(aws s3api list-object-versions --bucket my-bucket --query='{Objects: Versions[].{Key:Key,VersionId:VersionId}}' --output=json --profile=my-profile)" --profile=my-profile 
    {
        "Deleted": [
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/data-IOT1xReRSWuNLjW0es4SPg.dat",
                "VersionId": "FvX7ePtV5MOLK2cxsWE.smTwMgnLoFie"
            },
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/data-o29VZ5YOTYWL_pZ0aATn4g.dat",
                "VersionId": "rodR2lazpLXZf3p1cXrBEXnnQDRYDGRj"
            },
            ...
    }

    To delete all delete markers:

    % aws s3api delete-objects --bucket my-bucket --delete "$(aws s3api list-object-versions --bucket my-bucket --query '{Objects: DeleteMarkers[].{Key:Key,VersionId:VersionId}}' --output=json --profile=my-profile)" --profile=my-profile
    {
        "Deleted": [
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/data-o29VZ5YOTYWL_pZ0aATn4g.dat",
                "VersionId": "KFGMc_HXfMxG6vt7vIvIxFoAUny9rDWT",
                "DeleteMarker": true,
                "DeleteMarkerVersionId": "KFGMc_HXfMxG6vt7vIvIxFoAUny9rDWT"
            },
            {
                "Key": "tests-7H_N5XLAT2K_sW5aGZfM1g/data-IOT1xReRSWuNLjW0es4SPg.dat",
                "VersionId": "en7k.4jeTceJNvQhZm0PlJUNa6pZEQiL",
                "DeleteMarker": true,
                "DeleteMarkerVersionId": "en7k.4jeTceJNvQhZm0PlJUNa6pZEQiL"
            },
            ...
    }




    Sunday, 2 February 2025

    Introduction to Large Language Models (LLMs)




    Single-turn vs Multi-turn conversation


    ...

    Token

    • the smallest unit of text that the model recognizes
    • can be a word, a number, or even a punctuation mark
    • 1  (English) word has approximately 1.3 tokens

    Context Caching


    In large language model API usage, a significant portion of user inputs tends to be repetitive. For instance, user prompts often include repeated references, and in multi-turn conversations, previous content is frequently re-entered.

    To address this, Context Caching technology caches content that is expected to be reused on a distributed disk array. When duplicate inputs are detected, the repeated parts are retrieved from the cache, bypassing the need for recomputation. This not only reduces service latency but also significantly cuts down on overall usage costs.


    Billing


    The price of using some LLM is usually in units of per 1M tokens. 
    If 1 word has 1.3 tokens, let's see how many words is this: 1w : 1.3t = x : 10^6 => x = 10^6 / 1.3 = 769'230 words ~ 770k words. 

    Billing is usually based on the total number of input and output tokens by the model.

    If Context Caching is implemented, input billing per 1M tokens can further be split into two categories:
    • 1M tokens - Cache Hit (1M tokens that were found in cache)
    • 1M tokens - Cache Miss (1M tokens that were not found in cache

    Use Case: Chatbots


    Chat History

    LLMs don't have any concept of state or memory. Any chat history has to be tracked externally and then passed into the model with each new message. We can use a list of custom objects to track chat history. Since there is a limit on the amount of content that can be processed by the model, we need to prune the chat history so there is enough space left to handle the user's message and the model's responses. Our code needs to delete older messages.


    Retrieval-Augmented Generation (RAG)

    If the responses from the chatbot are based purely on the underlying foundation model (FM), without any supporting data source, they can potentially include made-up responses (hallucination). 

    Retrieval-augmented generation LLMs create a more powerful chatbots that incorporates the retrieval-augmented generation pattern to return more accurate responses.

    ...

    Resources:

    Tuesday, 28 January 2025

    Introduction to VertexAI


    Vertex AI

    • Google's managed machine learning platform
    • Designed to help developers, data scientists, and businesses:
      • build
      • deploy
      • scale machine learning models
    • Provides a comprehensive suite of tools for every stage of the machine learning lifecycle, including:
      • data preparation
      • model training
      • evaluation
      • deployment
      • monitoring

    Here’s an overview of what Vertex AI offers:

    Key Features of Vertex AI:

    1. Unified ML Platform: It consolidates various Google AI services into one integrated platform, making it easier to manage end-to-end workflows.

    2. Custom and Pre-trained Models:

      • You can train your custom machine learning models using your own data.
      • Alternatively, use Google’s pre-trained models or APIs for common AI tasks (e.g., Vision AI, Translation AI, and Natural Language AI).
    3. AutoML:

      • Offers an automated way to train machine learning models, making it accessible even to those without deep expertise in ML.
    4. Notebooks:

      • Managed Jupyter Notebooks are available for building and experimenting with ML models.
    5. Data Preparation and Labeling:

      • Tools for managing datasets, preparing data, and labeling it for supervised learning tasks.
    6. Training and Tuning:

      • Supports large-scale training with powerful infrastructure and features like hyperparameter tuning for optimizing models.
    7. Model Deployment:

      • Seamlessly deploy models to an endpoint for real-time predictions or batch processing.
    8. Model Monitoring:

      • Tracks the performance of deployed models, monitoring metrics such as prediction drift or latency.
    9. Integration with BigQuery and Google Cloud Services:

      • Easily access and analyze data stored in BigQuery and integrate it with other Google Cloud services.
    10. ML Ops Features:

      • Tools for managing and automating ML workflows, like pipelines and version control for reproducibility.

    Why Use Vertex AI?

    • Scalability: It handles infrastructure concerns so you can focus on model development.
    • Ease of Use: Tools like AutoML simplify machine learning for those with less technical expertise.
    • Cost-Effectiveness: Pay-as-you-go pricing lets you control costs.
    • Integration: Works seamlessly with Google Cloud services, making it a powerful choice for businesses already in the Google ecosystem.

    It’s ideal for both beginners looking for simplicity and experts needing advanced tools and customizability.


    Friday, 17 January 2025

    Introduction to Elasticsearch





    What is Elasticsearch?

    • An open-source analytics and full-text search engine.
    • Commonly used to enable search functionality for applications, such as blogs, webshops, or other systems. Example: in blog, search for blog posts, products, categories

    Capabilities of Elasticsearch:

    • Supports complex search functionality similar to Google:
      • Autocompletion.
      • Typo correction.
      • Highlighting matches.
      • Synonym handling.
      • Relevance adjustment.
    • Enables filtering and sorting, such as by price, brand, or other attributes.

    Advanced Use Cases:

    • Full-text search and relevance boosting (e.g., highly-rated products).
    • Filtering and sorting by various factors (price, size, brand, etc.).

    Analytics Platform:

    • Allows querying structured data (e.g., numbers) and aggregating results.
    • Useful for creating pie charts, line charts, and other visualizations.

    Application Performance Management (APM):

    • Common use case for monitoring logs, errors, and server metrics.
    • Examples include tracking web application errors or server CPU/memory usage, displayed on line charts.

    Event and Sales Analysis:

    • Analyze events like sales from physical stores using aggregations.
    • Examples include identifying top-selling stores or forecasting sales using machine learning.

    Machine Learning Capabilities:

    • Forecasting:
      • Sales predictions for capacity management.
      • Estimating staffing needs or server scaling based on historical data.
    • Anomaly detection:
      • Identifying significant deviations from normal behavior (e.g., drop in website traffic).
        • machine learning learns the “norm” and let you know when there is an anomality, i.e. when there is a significant deviation from the normal behavior.
      • Automates alerting for unusual activities without needing manual thresholds.
      • We can then set up alerting (email, Slack) for this and be notified whenever something unusual happens

    How Elasticsearch Works:

    • Data is stored as documents (JSON objects), analogous to rows in a relational database.
    • Each document has fields, similar to columns in a database table.
    • Uses a RESTful API for querying and interacting with the data.
    • Queries are written in JSON, making the API straightforward to use.

    Technology and Scalability:

    • Written in Java and built on Apache Lucene.
    • Highly scalable and distributed by nature, handling massive data volumes and high query throughput.
    • Supports lightning-fast searches, even for millions of documents.

    Community and Adoption:

    • Widely adopted by large companies and has a vibrant community for support and collaboration.


    Index Templates


    Deletion:

    curl -u "user:pass" -X DELETE "https://elasticsearch.my-corp.com:443/_index_template/index_template_name"


    Thursday, 9 January 2025

    ELK Stack Interview Questions




    Elasticsearch (ES)





    Kibana


    • How to install Kibana on bare metal?
      • How to install Kibana in k8s cluster?
    • What are Dashboards?
    • What are Alerts?
    • How to back up and Elastic objects like dashboards and alerts? How to restore them in another Elastic instance?
    • TBC