In this article, we are going to explore the essential role of callbacks in controlling and optimizing the training of deep learning models, exploring their significance, usage, and how to implement them using Keras. This guide will help you master the art of callbacks and optimize your training process.

Understanding Callbacks

Callbacks are a widely used concept in most programming languages. It is a concept of passing functions/methods or pieces of code as an argument to other functions or methods. These functions are intended to be executed at a later time, often in response to a specific event or condition, such as completing a task or handling user interaction.

Callbacks in Deep Learning work on the same principle enabling us to monitor, adapt, and optimize the training process of our models. Keras provides built-in API to their callback object which has some amazing callback methods used for a variety of tasks. They come to the rescue when we need to implement real-time actions based on the model’s performance. The possibilities with callbacks are endless, and they play a pivotal role in ensuring that your Deep Learning model evolves into a powerful asset.

These Callback methods can be passed to Keras methods including fit, evaluate, and predict as a list in order to hook into the various stages of the model training and inference lifecycle.

model.fit(
    train_data, 
    epochs=2, 
    batch_size=32,
    callbacks=[early_stopping]
)

This was a basic introduction to Keras Callbacks, Now let’s dive into these callbacks and their practical implementations.

Built-in Callbacks in Keras

Keras provides some built-in Callbacks that help to boost the training process. This includes early stopping, Model Checkpoint, and Learning Rate Scheduler. Let's check out all the amazing callbacks from Keras and their usage.

EarlyStopping

One of the most common challenges in training machine learning or deep learning models is overfitting. EarlyStopping is a callback that allows us to address this issue effectively. By monitoring the model’s performance on a validation set, this callback can halt the training process when no improvement is observed, preventing overfitting and saving valuable time and resources.

from keras.callbacks import EarlyStopping

early_stopping = EarlyStopping(
    monitor="val_loss",
    min_delta=0,
    patience=5,
    verbose=0,
    mode="auto",
    baseline=None,
    restore_best_weights=False,
    start_from_epoch=0,
)

Arguments

• monitor: This parameter specifies the quantity to be monitored for improvements. It can be metrics like “val_loss,” “val_accuracy,” etc.

• min_delta: Minimum change required in the monitored quantity to be considered for improvement. If the change is less than min_delta, it won’t count as an improvement.

• patience: The number of epochs with no improvement after which training will be stopped. For instance, if patience=5, training will stop if there is no improvement for 5 consecutive epochs.

• verbose: It controls the verbosity mode. Use 0 for silent mode, which means no messages are displayed. Set it to 1 for mode where messages are displayed when the callback takes action.

• mode:{"auto", "min", "max"}. In min mode, training stops when the monitored quantity stops decreasing; In "max" mode, it stops when the monitored quantity stops increasing; In "auto" mode, the direction is automatically determined based on the name of the monitored metric.

• baseline: Baseline value for the monitored quantity. Training will stop if the model doesn’t show improvement over this baseline.

• restore_best_weights: When set to True, it restores model weights from the epoch with the best value of the monitored quantity. If set to False, the model uses weights obtained at the last step of training, regardless of performance relative to the baseline.

• start_from_epoch: This parameter allows you to specify the number of warm-up epochs before monitoring for improvement. During this period, no improvement is expected, and training won’t be stopped.

ModelCheckpoint

Saving the model’s weights at regular intervals is crucial to ensure that your progress is not lost. ModelCheckpoint comes to the rescue by saving the model’s weights in a structured manner. This allows you to resume training from where you left off, ensuring continuity in your Deep Learning journey.

from keras.callbacks import ModelCheckpoint 

model_checkpoint = ModelCheckpoint(
    filepath,
    monitor: "val_loss",
    verbose: 0,
    save_best_only: False,
    save_weights_only: False,
    mode: "auto",
    save_freq="epoch",
    options=None,
    initial_value_threshold=None,
    **kwargs
)

Arguments

• filepath: file path to save the model weights.

• monitor: This parameter specifies the quantity to be monitored for improvements. It can be metrics like “val_loss,” “val_accuracy,” etc.

• verbose: Controls the verbosity mode. Use 0 for silent, and 1 for displaying messages when the callback takes action.

• save_best_only: For True, the callback saves only the best model (based on the monitored quantity). If set False, it saves the model at every checkpoint according to the specified frequency.

• save_weights_only: For True, it saves only the model’s weights. If set to False, it saves the entire model.

• mode:{"auto", "min", "max"}. In min mode, training stops when the monitored quantity stops decreasing; In "max" mode, it stops when the monitored quantity stops increasing; In "auto" mode, the direction is automatically determined based on the name of the monitored metric.

• save_freq: Determines how often to save the model. It can be set to “epoch” (saves at the end of each epoch) or an integer value (saves every n batches or steps).

• options: Additional options for saving the model, such as compression options.

• initial_value_threshold: To skip some initial epochs based on some initial value for monitored quantity, so as to save the model when the monitored quantity when threshold exceeds

• **kwargs: Additional keyword arguments that are passed to the underlying object for backward compatibility.

TensorBoard

Visualization is key to understanding your model’s behavior. TensorBoardCallback helps you create visual representations of training progress. By monitoring metrics like loss and accuracy, you can spot issues and optimize your model training effectively.

from keras.callbacks import TensorBoard 

tensorboard_callback = TensorBoard(
    log_dir="logs",
    histogram_freq=0,
    write_graph=True,
    write_images=False,
    write_steps_per_second=False,
    update_freq="epoch",
    profile_batch=0,
    embeddings_freq=0,
    embeddings_metadata=None,
    **kwargs
)

Arguments

• log_dir: Path to the directory where the TensorBoard logs will be saved

• histogram_freq: It controls how frequently histograms of your tensors are computed for visualization. Set to 0 for no histograms or a positive integer to define the frequency.

• write_graph: True, saves the computation graph for visualization in TensorBoard.

• write_image: True, save images like model architecture graphs and feature maps.False, these images won't be saved.

• write_steps_per_second: True, records the number of steps per second, giving you insights into training speed. Useful for performance monitoring.

• update_freq: Determines how often the logs are updated. It can be set to "epoch" (updating once per epoch) or a positive integer for a different frequency.

• profile_batch: allows you to profile the performance of your model. Setting 0 will disable profiling, while a positive integer specifies the batch to profile.

• embeddings_freq: Set to a positive integer to determine how frequently the embeddings are computed and visualized.

• embeddings_metadata: A file path to metadata that describes your embeddings, providing context for visualization.

• **kwargs: Additional keyword arguments that are passed to the underlying object for backward compatibility.

ReduceLROnPlateau

Learning rate optimization is a critical aspect of training deep learning models. ReduceLROnPlateau automatically adjusts the learning rate when the validation loss plateaus. This dynamic adaptation ensures your model converges efficiently.

from keras.callbacks import ReduceLROnPlateau 

reduce_lr = ReduceLROnPlateau(
    monitor="val_loss",
    factor=0.1,
    patience=10,
    verbose=0,
    mode="auto",
    min_delta=0.0001,
    cooldown=0,
    min_lr=0,
    **kwargs
)

Arguments

• monitor: This parameter specifies the quantity to be monitored for learning rate reduction. It can be metrics like “val_loss,” “val_accuracy,” etc.

• factor: The factor by which the learning rate will be reduced. For example, if is set to 0.1, the learning rate will be reduced to 10% of its current value when the condition is met.

• patience: The number of epochs with no improvement after which the learning rate will be reduced.

• verbose: Controls the verbosity mode. Use 0 for silent, and 1 for displaying messages when the callback takes action.

• mode:{"auto", "min", "max"}. In min mode, training stops when the monitored quantity stops decreasing; In "max" mode, it stops when the monitored quantity stops increasing; In "auto" mode, the direction is automatically determined based on the name of the monitored metric.

• min_delta: Minimum change required in the monitored quantity to be considered for improvement. If the change is less than min_delta, it won’t count as an improvement.

• cooldown: The No. of epochs to wait after a learning rate reduction before resuming normal monitoring. During this “cooldown” period, learning rate changes won’t occur.

• min_lr: The lower limit on the learning rate. If the learning rate falls below this value, it won’t be reduced further.

• **kwargs: Additional keyword arguments that are passed to the underlying object for backward compatibility.

LearningRateScheduler

Sometimes, standard learning rate schedules may not be suitable for your specific problem. LearningRateScheduler gives you the flexibility to craft custom learning rate schedules tailored to your model’s needs.

from keras.callbacks import LearningRateScheduler

def custom_schedule(epoch):
    if epoch < 10:
        return 0.001
    else:
        return 0.0001

lr_scheduler = LearningRateScheduler(custom_schedule)

Arguments

• schedule: A custom function with the user-defined rule that takes an epoch index (integer) and current learning rate (float) as parameters and returns the new learning rate.

• verbose: Controls the verbosity mode. Use 0 for silent, and 1 for displaying messages when the callback takes action.

TerminateOnNaN

It’s crucial to maintain data integrity during training. TerminateOnNaN is a simple yet effective callback that stops training if NaN values appear in your model’s outputs, acting as a data guardian to keep your training on track.

from keras.callbacks import TerminateOnNaN

terminate_on_nan = TerminateOnNaN()

Creating Custom Callbacks

While pre-built callbacks are incredibly useful, there are situations where you might need a callback tailored to your specific needs. Fortunately, Keras provides a straightforward way to create custom callbacks. Keras has an Abstract base class that can be used to build new callbacks. This base class is parent to each callback method in Keras.

keras.callbacks.Callback()

Let’s dive into how you can do it:

Step 1: Define Your Custom Callback Class

You can create a custom callback by defining a Python class that inherits the Keras base class. This class should include methods for the events you want to monitor and customize.

Here’s a basic structure to get you started:

class CustomCallback(keras.callbacks.Callback):
    def on_train_begin(self, logs=None):
        # Called at the beginning of training
        pass
    def on_epoch_end(self, epoch, logs=None):
        # Called at the end of each epoch
        pass    # You can add more event-specific methods here

Step 2: Implement Your Callback Logic

Within your custom callback class, you can implement your specific logic for each callback event. For example, you might want to save the model’s weights at the end of every epoch. Here’s how you can achieve this:

class CustomCallback(keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs=None):
        model.save_weights(f"model_weights_epoch_{epoch}.h5")

This code will save the model’s weights to a file at the end of each epoch.

Step 3: Incorporate Your Custom Callback

Once you’ve defined your custom callback, you can use it by passing an instance of your class to the callback parameter when fitting your model. For example:

model.fit(
    x_train, y_train,
    batch_size=32, epochs=10,
    callbacks=[CustomCallback()]
)

Your custom callback will now be executed during training, and you can tailor it to perform actions that suit your project’s needs.

Creating custom callbacks gives you the flexibility to track and manipulate various aspects of the training process, making it a powerful tool in your deep learning toolkit.

You have seen several Keras callback methods for different tasks. These callback methods will boost your training process, saving time and resources. So, what are you waiting for? Start implementing these powerful Keras callbacks in your projects and witness the transformation of your deep learning journey. Your AI-powered solutions are just a callback away from greatness!

The HTML5 element <picture> tag is intended to provide more flexible and highly functional responsive image. Rather than loading a single image and adjusting it to match all potential viewport widths and layouts, the picture element loads multiple images of varying sizes and resolutions, selecting the best fit for various scenarios.

The picture tag can have "source" attributes and one img attribute to provide an alternative copy of a picture for various browsers and displays. The browser will evaluate each child's <source> element and select the best match from among them. If no matches are discovered, or if the browser does not support the <picture> element, the URL specified in the <img> element's src property is used. The selected picture is then shown in the <img> element's area.

Let's see an example

<picture>
  <source media="(min-width:769px)" srcset="img-medium.png">
  <source media="(min-width:465px)" srcset="img-small.png">
  <img src="img.jpg" alt="Flowers" style="width:auto;">
</picture>

srcset, media, and other basic characteristics of each <source> element in the picture tag are analyzed by the browser and find the best compatible picture for the current layout and capabilities of the display device. The picture tag helps us to select and choose the best-fit image for different device sizes and browser compatibility. It will help us save bandwidth and will speed up the page load times by loading these best-fit images for the viewer's display.

Attributes of Picture Tag

1. srcset

The srcset attribute is used to provide a list of potential pictures based on their size. It consists of a list of image descriptions separated by commas. Each image descriptor consists of

• Image URL
• Width Descriptor (such as 300w, 500w)
• Pixel Density Descriptor (such as 1x, 2x) to deliver a high-resolution picture for high-DPI devices.

<picture>
  <source srcset="pic.png 768w, pic-1.5x.png 1.5x">
  <source srcset="pic-480.png 480w, pic-480-2x.png 2x">
  <img src="pic-320.png">
</picture>

2. Media

For each <source> the element that the user agent will examine, the media attribute specifies a media condition (similar to a media query). If the media condition of the <source> evaluates as false, the browser skips it and moves on to the next element within <picture>.

<picture>
  <source srcset="mdn-logo-wide.png" media="(min-width: 600px)">
  <img src="mdn-logo-narrow.png" alt="MDN">
</picture>

3. Type

The type attribute specifies a MIME type (Multipurpose Internet Mail Extensions) for the resource URL(s) in the srcset attribute of the <source> element. The <source> element is skipped if the user agent does not support the specified type. This will help us to load an image with high quality and low size so that the loading speed of the image will be less.

<picture>
  <source srcset="pic.png" type="image/png">
  <source srcset="pic.avif" type="image/avif">
  <source srcset="pic.webp" type="image/webp">
  <img src="pic.jpg" alt="photo">
</picture>

Browser Compatibility

You can check in detail in caniuse

Prerequisites :

• Python
• Git
• Pip
• VSCode (You can use any code editor of your choice)

Creating a Simple Flask App

Open a folder in your favorite code editor and create a virtual environment using pip in terminal using the command written below. Once the virtual environment setup is completed now we have to activate it using a script. Before executing the activation script make sure you're in the folder that contains the virtual environment you just created.

Windows

python -m venv <name_for_virtualenv>
<name_for_virtualenv>\Scripts\Activate.ps1

Linux/MacOs

python3 -m venv <name_for_virtualenv>
source  <name_for_virtualenv>/bin/activate

Now you can see (name_for_virtualenv). if you don't see this then your venv is not activated. Once the virtual environment is activated now we will install the necessary python packages for our Flask app. we will need to install Flask and Gunicorn engine. Gunicorn is a Python HTTP server for WSGI applications.

pip install flask gunicorn

You can check the list of python packages installed in the virtual environment using a code i.e.

pip list

You will see a list of packages installed in your virtual environment. pip and setuptools are the base packages that are default in a new virtual environment. Other packages include flask, gunicorn, and some other lists that are by default installed with flask.

Flask App

Now you're ready to start with your flask app. Create a file i.e. main.py and paste the following code

from flask import Flask

app = Flask(__name__)

@app.route("/")
def home_view():
        return "<h1>My Flask App</h1>"

if __name__ == "__main__":
  app.run()

Now create a file with the name Procfile and make sure the Procfile has no extension and paste the following code

web: gunicorn main:app

Your flask app is ready to deploy. you can check your flask app by running a command in the terminal i.e.

python main.py

you will see an address of localhost where your site is currently running. You can check it in your browser. Paste the LocalHost URL in the browser.

Now we will create a text file for all the necessary python package names and versions that we'll need on our server. We don't need to write these names manually we'll do it using a built-in command i.e.

pip freeze>requirements.txt

This will create a file of name requirements with all the packages and their versions. below is the list of packages.

Setting Up GitHub Repository

Now create a new GitHub repository with an appropriate name. After you created an empty repository you will get a page

On this page, all the necessary commands are already provided you can use these commands to push your code to GitHub. First Open your terminal in the project's directory and write the following commands step by step to upload your project to GitHub. Before writing these commands make sure you have installed git in your operating system

Initialize your Folder to the Repository:

git init

Add gitignore file

Now add gitignore to ignore the virtual environment folder because we don't need to upload it. Create a file using the name .gitignore in the project's directory. Add the folders that need to be ignored we're ignoring environment directed so write env/ in the .gitignore file.

Add file to Git

git add .

Use the above command to add all the files to git

Add comment

git commit -m "message here"

the above comment will set a message to the commit so it will be easy for backtracking. Make sure the comment describes your changes.

Change Branch to main

git branch -M main

The above command is one time you don't need to write this command on every change. When you initialize a directory the default branch is master so we need to change it to the appropriate one.

Linked to GitHub

git remote add origin https://github.com/<username>/<repository-name>.git

The above command is also one-time. This command is used to link your local repository to the GitHub repository. You can find this link in your GitHub repository.

Push to GitHub

git push -u origin main

The above command is used to upload your code to GitHub. After successfully uploading refresh the GitHub page to see your files on your repository.

Deploy on Heroku

Login/Signup to your Heroku account. Once you have logged in to your Heroku Account you will find a button with text new. Click on the button you will get a dropdown menu. In the dropdown menu click on create a new app. then provide the appropriate name for your app and click on 'create app'.

when you have created the app you'll redirect toward the main dashboard. From Dashboard click on connect to GitHub. Then a Menu will appear where you can search for the repository you want to deploy on Heroku. Now select Enable Automatic Deploys and click on deploy branch to deploy your project. After successful deployment, a message will show i.e. "Your app was successfully deployed". Now you can see your site online by clicking on the open app button at top of the dashboard.

Congratulations you have successfully deployed your app on Heroku. whenever you push a change on GitHub Heroku will automatically deploy it and changes can be seen live.

Have you ever tried the switch statement in Python? If you have been programming in other languages such as C++, C#, Java, etc, you will have used the switch statement, but when you shifted to Python, you noticed there is no switch statement in Python.

But here's some good news. Python's new release, i.e., Python 3.10.5, provides switch statements to Python users. A switch statement has been added to Python in the form of match statements and case statements of patterns with corresponding actions. Pattern matching allows programs to extract information from complex data types, branch based on data structures, and perform specified actions based on various data types.

Before proceeding to Syntax & Examples, make sure you have upgraded to the latest version of Python, i.e., Python 3.10.5, by downloading it from Python's Official Website

Match General Syntax

The syntax is pretty much the same as other programming languages use. Instead of the "switch" keyword, we use the "match" keyword in Python.

The generic syntax of pattern matching is:

match subject:
    case 1:
        # code here
    case 2:
        # code here
    case 3:
        # code here
    case _:
        # _ is used for default. if in case the value is not matched with any case default will execute 
        # code here

• First, a variable of various types will be passed to match the operator.
• Then, the match statement will evaluate the value of the variable.
• After that, the variable value will be compared with each case from top to bottom until matched with a suitable case.
• Then the code inside the matched case will be executed.
• If there is no case-matched, then a wildcard "_". If a wildcard is provided and provided cases do not match, then the wildcard will be used as a matched case. (Such as the default case in various programming languages).

Match Example

Let us check a simple example of pattern matching: A value, the subject, is matched to several literals, the patterns. The subject of the match statement in the example below is status. Each of the case statements is a pattern, with literals representing request status codes. After a match, the case's corresponding action is carried out:

num1, num2 = 5, 6
op = input("Enter Operator")
result = 0
match op:
     case "+":
          result = num1+num2
     case "-":
          result = num1-num2
     case "/":
          result = num1/num2
     case "*":
          result = num1*num2
     case _:
          print("Operator Not Found")

If op is selected "+", the sum operation will be performed. If op is selected "%", the case statement with will match as a wildcard, and “Operator Not Found” will be printed. Note the last block: the variable name, , acts as a wildcard and insures the op will always match. The use of _ is optional.

You can combine several literals into a single pattern using | ("or"):

case "+" | "sum" | "add":
    result = num1 + num2

Match Without the wildcard, how would it behave?

If we remove the last case block from the above example, it becomes:

num1, num2 = 5, 6
op = input("Enter Operator")
result = 0
match op:
     case "+":
          result = num1+num2
     case "-":
          result = num1-num2
     case "/":
          result = num1/num2
     case "*":
          result = num1*num2

A match may not exist without the use of _ a case statement. If there is no match, then the behavior is a no-op. For example, if op is set to "%", a no-op occurs.

Matching with literals and variables

Patterns can resemble unpacking assignments, and they can also be used to bind variables. A data point in this example can be unpacked to its x- and y-coordinates:

x, y = 1, 2

point = (1, 0)

match point:
     case (0, 0):
          print("Origin")
     case (0, y):
          print(f"Y={y}")
     case (x, 0):
          print(f"X={x}")
     case (x, y):
          print(f"X={x}, Y={y}")
     case _:
          print("not a point")

The first pattern has two literals, (0, 0), and is an extension of the literal pattern mentioned above. The next two patterns combine a literal and a variable, with the variable binding a subject value (point). The fourth pattern catches two variables, it is essentially identical to the unpacking assignment (x, y) = point.

Matching with Class or Class Object

If you use classes to structure your data, you can use the class name followed by an argument list similar to a constructor as a pattern. This pattern can grab class characteristics and store them in variables:

class Point:
    x: int
    y:int

x, y = 1, 1

point = (0, y)

match point:
    case Point(x=0, y=0):
        print("Origin is the point's location.")
    case Point(x=0, y=y):
        print(f"Y={y} and the point is on the y-axis.")
    case Point(x=x, y=0):
        print(f"X={x} and the point is on the x-axis.")
    case Point():
        print("The point is located somewhere else on the plane.")
    case _:
        print("Not a point")

Or you can simply match using class object also

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

x, y = 1, 1

point = Point(0, y)

match point:
    case Point(x=0, y=0):
        print("Origin is the point's location.")
    case Point(x=0, y=y):
        print(f"Y={y} and the point is on the y-axis.")
    case Point(x=x, y=0):
        print(f"X={x} and the point is on the x-axis.")
    case Point():
        print("The point is located somewhere else on the plane.")
    case _:
        print("Not a point")

An if clause, known as a "guard," can be added to a pattern. If the guard is false, the match proceeds to the next case block. It's worth noting that value capture occurs before the guard is evaluated:

match point:
    case Point(x, y) if x == y:
        print(f"The point is on the diagonal Y=X at {x}.")
    case Point(x, y):
        print(f"Point is not on the diagonal.")

Matching with Complex pattern

You have seen in the above examples we have used wildcard alone. but we can also use it in a more complex manner:

match response:
    case ('success', code, 200):
        print("Response received.")
    case ('error', code, _):
        print(f"An error {code} occurred while responding.")

Stay tuned and follow for new updates

lambda arguments: expression

It can take any number of arguments, but can only have one expression. Lambda Function Example Below is a simple lambda function example.

x = lambda a, b: a + b
print(x(5,6))

Output:
11

Uses of Lambda Functions

Lambda function is better shown when they are used as an anonymous function inside another function. Let's suppose you have a function definition that takes one argument, and that argument will be powered by an unknown number:

def power(n):
     return lambda a: a ** n

Use that function definition to make a function that always squares the number you send

square = power(2)
print(square(3))

Or similarly, we can generate tons of other functions from that single function such as

cube = power(3)
print(cube(3))

IVTimes = power(4)
print(IVTimes(3))

The idea of Python programming was conceived in the late 1980s by Guido van Rossum. In late 1989, He was looking to work on a Hobby project that will keep him busy near Christmas days. So he decided to write an interpreter for the language he had been thinking of.

Python, as a successor of ABC language, with the ability of Exceptional Handling and interfacing with the Amoeba OS. Since he had a wide experience with the implementation of the interpreted language in the ABC group at CWI. (Centrum Wiskunde & Informatica is a research center in mathematics & theoretical computer science). So as he has written he learned a lot about language design at CWI and this is the main origin of most Python features.

Python 0.9.0, First Release of today's top-level Programming language

Python was first released on 20th Feb 1991 as Python 0.9.0 at alt. source. During its first release, It was already enriched with exceptional handling, core data types, and functions. Python was object-oriented from its first release and has a module system at that time also. In Jan 1994, Python version 1.0 has released with major new features like lambda, filter, and map and reduces. which were more functional programming tools.

Python 2.0. A major update in the history of Python Programming

Python version 2.0 was released on October 16, 2000, with many major new features. that includes a cycle-detecting garbage collector for memory management and support for Unicode. Yet, the most important change was the development process in Python 2.0. with a shift to a more transparent and community-backed process.

Python 3.0, The latest version that created history in Python Programming

Python version 3.0 has released on December 3, 2008, after a long period of testing. The new version was incompatible with the previous 2. x line releases. In this new version, many details such as Built-in objects have changed. And removed many deprecated features.