Yashints | Blog

🖥️ Use local AI to supercharge your coding skills

Fri, 07 Jun 2024 00:00:00 GMT

Are you a developer who wants to use AI in their day to day coding but don’t have access to tools like GitHub Copilot, GitHub Copilot Workspace or Cloud based services like Azure OpenAI? If yes, read on and you’re going to have super powers by the end of this post.

Why bother?

If you’re wondering what is all the fuss and why should I bother, I have to tell you AI is moving like a speed train and if you don’t jump in, you’ll be left behind and it would be very hard to catch on later. AI can help you enhance your coding skills, increase your productivity, and help you learn a great deal more than you do in your job.

Here are some reasons why any developer needs to use AI:

Automated code generation: AI can generate code and save you time and effort. You can get suggestions with snippets of code and even get the rest of your code completed as soon as you have some of it written. This can speed up your coding and development process by a great deal.
Code review: AI can help to review code and identify potential issues, bugs, security vulnerabilities and code smells before they even become problems. This can improve the overall quality of your code and products and reduce time spent fixing bugs after go live.
Personalised learning: AI can provide personalised learning experience for you and help improve your skills and knowledge in a way which is tailored to your needs and learning style.
Natural language (NLP): In addition to all of the previous points, you get to use your natural language to interact with AI using prompts, voice etc which makes the whole process even faster.

Setup - let’s get cracking

Enough talk, let’s get into the action. You will need these before you can leverage AI in your development environment:

Ollama which is an open source project allowing you to interact with LLMs locally. If you don’t have it setup, use my earlier post to get setup. We also need to have a model installed which supports code generation.
VS Code which is also an open source code editor which has billions of users globally. You can use JetBrains if you are using that already BTW.
Continue which is an extension which allows you to leverage LLMs in your code editors using cloud or local hosted models.

Install the code generation model

Go into the environment which you have Ollama installed and install the starcoder model. You can get the 3b parameter version or if you’re adventurous like me, get the 7b version which is roughly 4GB.

Copy

ollama pull starcoder2:7b

And you should see the model being pulled and installed.

Install the extension

First you need to install Continue extension in VS Code. You can click on the link from here or go into your VS Code extensions and get it installed.

Configuring the extension

Once you have both of those installed, you can go ahead and configure your extension to use your local Ollama instance. Open the config.json settings for Continue using Ctrl + Shift + P, then typing > continue: open config.json. Keep in mind that Continue supports a whole host of providers other than local ones. Go ahead and add the models you want to the config file. Do not forget to change the IP address to match your own instance if it is not available on localhost.

Copy

{
  "models": [
    {
      "title": "Llama 3",
      "provider": "ollama",
      "model": "llama3",
      "apiBase": "http://localhost:11434/"
    },
    {
      "title": "Starcoder 7b",
      "provider": "ollama",
      "model": "starcoder2:7b",
      "apiBase": "http://localhost:11434/"
    }
  ],
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Starcoder 7b",
    "provider": "ollama",
    "model": "starcoder2:7b",
    "apiBase": "http://localhost:11434/"
  },
  "allowAnonymousTelemetry": true,
  "embeddingsProvider": {
    "provider": "transformers.js"
  }
}

Testing

Let’s test our setup, go ahead and open a file in which you have any code written. Select a code block and press Ctrl + L which opens the chat window, type Explain this code to me in a few sentences and press enter. You should see the explanation getting generated in the chat window:

Using tab for auto completion

You can use tab to complete code in your code file. Create a new file called index.js, enter below comments:

Copy

// a function which calculates fibonacci sequence
// and returns the nth number in the sequence

Go to the next line and wait until your see the function name getting generated, if it didn’t type the function name and wait, it should generate the function and show you in highlighted grey text:

Press Tab and the code should be persisted.

Refactor code

When you’re writing code, you could ask AI to refactor the code for you. Go ahead and clear the content of the file and replace it with below:

Copy

function calculateTotal(items) {
  let total = 0;
  for (let i = 0; i < items.length; i++) {
    total += items[i];
  }
  return total;
}

Now press Ctrl + I and type refactor this code in the prompt and press Enter. Wait for the model to do its thing and you should see the result which you can accept or reject:

Ask questions about your codebase

You can also ask questions about your codebase by pressing Ctrl + L and typing @codebase and then your question.

Use documentation as context for your questions

You can use documentation when interacting with AI which adds more accuracy to the response. The team have done a fantastic job and have pretty much any language or framework you can think of linked. You can use the @docs and then select your preferred documentation, or just simply type the doc name if you know it.

Let’s open the chat window using Ctrl + L, then type @react and then How do I update the page based on a parameter? then press Enter.

Perform actions with slash

You can kick off an action using the slash in the chat window, this could be edit, comment, share, commit, test (which I had in the config and shows you how to do custom ones) or any predefined custom command.

I will use the test command, select the code in your file and press Ctrl + I, then /test and finally Enter.

Use files as context

You can use files in your context by using the @files tag. Simply type that in the search box and you will get prompted to select a file, or you can type the file name directly. Then type your question and press Enter.

Understand errors in terminal

When you get any errors or issues in your terminal, select the error text and press Ctrl + Shift + R and you will get an explanation:

Summary

This was another exciting step towards enabling everyone around the world to use AI in their day to day tasks without having to pay any fees or be dependant to any cloud provider. I hope you enjoyed and could get the setup up and running, and stay tuned for more AI content.

🔮 fabric, augmenting humans using AI

Sat, 01 Jun 2024 00:00:00 GMT

If you have worked with OpenAI or any other Generative AI model aka LLMs, you would know that writing a good prompt is a very key part of how we interact with those models. However, it’s often ignored which most probably means people are not getting the most out of these super capable models.

The why

The most important part of interacting with LLMs is writing a good prompt, and that’s why it is important to make sure not only you know how to write a good prompt, but also are aware of what tools are available which might help you do just that. I wrote this post to introduce fabric to you all since it has changed the way I personally use my local models and I can’t believe how did I not know about it before.

Introducing fabric

fabric is an open source tool which augments humans using AI. I know it sounds a little fancy, but believe me when you get to know its features and usefulness, you would totally be onboard with the definition.

In short, fabric is a tool which not only has many patterns (a comprehensive prompt for a specific purpose) already baked in, but also allows you to use them with LLMs like OpenAI or even local LLMs (read my last post on how to install Ollama) if you have them handy. Furthermore, you can add your own patterns if you like to the mix too and contribute back to the community if your pattern deemed useful to others.

Enough talking, let’s get it installed and show you some cool features.

Prerequisites

You would need a few things installed to be able to seamlessly install fabric. I am using WSL to install it, but you can install it in a linux distro of your choice.

pipx: fabric uses pipx to install itself, so get it installed by running:

Copy
```
sudo apt install pipx
```
python3-dev: Since it is using Python extensions, you would need to install the header files:

Copy
```
sudo apt-get install python3-dev
```
ffmpeg: Fabric depends on this library and libavcodec-extra when handling multimedia:

Copy
```
sudo apt-get install ffmpeg libavcodec-extra
```

Installing fabric

You wouldn’t believe how simple the installation process is. First navigate to a folder where you intend to clone the repository, clone the repo and then navigate into the fabric folder:

Copy

cd /where/you/keep/code
# if the path is a mounted drive you might have to run this with sudo
git clone https://github.com/danielmiessler/fabric.git
cd fabric

Now use pipx to install fabric:

Copy

pipx install .

Once installed, call the setup function where you have provide four API keys for OpenAI, Claude APIs, Google and YouTube APIs which you can get them from their respective consoles.

Copy

fabric --setup

When you are finished, you need to exit bash and go back in to have access to fabric:

Copy

exit

And in your terminal:

Copy

bash

Now you can run fabric and see the list of available patterns:

Copy

fabric --list

Which should show you the list of available patterns.

Features

Now let’s explore some cool features that will change they way you use LLMs forever. But first, we need to set our default model to make sure we are not charged by OpenAI or other available online models:

Copy

ollama list
# replace the name of the model with whatever you have available
fabric --setDefaultModel llama3:latest

Now we’re ready to go. First off, let’s use one of the features which I use almost every day. I have subscribed to many YouTube channels be it tech or not, and many of those post long videos on a daily basis. You can summarise the whole video using one command:

Copy

yt --transcript https://youtube.com/watch?v=uXs-zPc63kM | fabric --stream --pattern extract_wisdom

We’re using the extract_wisdom pattern here, but you can use any other pattern you want. The API key you provided for YouTube is used here to get the transcript and pass it to the LLM using the pattern you have chosen. Super cool and fast response makes this a really great setup.

Let’s try something different:

Copy

echo "An idea that coding is like speaking with rules." | fabric -sp write_essay

Here we’re passing an idea and ask fabric to call our model and ask for an essay. The result will truly amaze you.

Taking it to next level

There are two library in macOS called pbcopy and pbpaste which allow you to copy text from and paste text into your terminal, but at large. Fortunately we can leverage a library which allows us to do exactly that in WSL or any other linux distro called xclip. Let’s get it installed and running:

Copy

sudo apt install xclip

nano ~/.bashrc

Now paste these two lines at the end of the file:

Copy

alias pbcopy='xclip -selection clipboard'
alias pbpaste='xclip -selection clipboard -o'

Exit nano with Ctrl + X, then enter Y and press Enter. Last we need to refresh our terminal:

Copy

source ~/.bashrc

We’re good to go. Navigate to any site which has a long article, I use this one talking about interest rates in Australia and what might happen in June. Copy all the text in that page and then type this in your terminal:

Copy

pbpaste | fabric -sp extract_insights

What next

There are many cool features you might be interested in, but I just highlight two more which I think are the most practical. Setting a context, and adding custom patters. Setting a context is done by adding a markdown file called context.md in the ./config/fabric/ directory to add context to your pattern. This could be anything you like, for example you might add your goals or the essence of your existence and why you do what you do day to day.

From the custom patterns perspective you can always add your custom patterns which you do not want to upload to fabric in the ~/.config/custom-fabric-patterns directory. However, whenever you want to use them with fabric you need to move them into the ~/.config/fabric/patterns directory. Just beware that any time you update fabric, the folder get updated and every custom pattern will be deleted, so you need to copy them from the custom folder before using them.

Summary

Hope you now have a good cause to go setup fabric and spend a little bit more time with your local LLMs and get better at writing prompts and extracting information or getting what you need faster and more effectively. Stay tuned for more AI content 👋🏽.

🥇 Run your generative AI model locally 🥇

Tue, 28 May 2024 00:00:00 GMT

Many learners ask me how they can get access to Azure OpenAI and Generative AI since they don’t have access to an enabled subscription. I knew there are open source tools which allow people to work with these models, but I never had a use case to go and try this out.

🩺 The need

I have a kid in high school and she wanted to give ChatGPT a try for a task she was working on, I was afraid she would use it to get the answer right away instead of figuring out how to solve the problem. This forced me to look into how I can have a better control over how she could use LLMs without “cheating”.

That’s when I stumbled upon Ollama, a great cross-platform, open source project which happens to be the easiest way you can get an engine running with many available models. Although their Windows version is experimental and in development, you can install it on Windows Subsystem for Linux aka WSL.

🖥️ Prerequisites

Before we start our process, you need to know that you will need a system which has a decent GPU since running AI locally demands that. This could even be your laptop which has a graphic card, or even better your gaming PC with that crazy spec you spent that much money on.

You most likely need docker installed as well since I will use it to run the web UI, but this is not mandatory.

⚙️ Installation

Of course you need WSL enabled and a distro installed, but don’t worry as soon as you enable Virtual Machine Platform and Windows Subsystem for Linux in your windows features, all it takes is a few commands:

Copy

wsl -- install -d ubuntu
wsl --update
wsl --set-default-version 2

Before we install any packages, it’s always a good idea to update the current ones, this becomes more important if you have had WSL enabled for a while.

Copy

sudo apt update && sudo apt upgrade

Now it’s time to install Ollama:

Copy

curl -fsSL https://ollama.com/install.sh | sh

This will take a while to finish, so get a coffee and be patient. Once it is installed it will start the Ollama engine and shows you it’s running. To test it you can open a browser and navigate to http://localhost:11434 which should show you it’s running:

Now you can pull down a model and interact with it:

Copy

ollama run llama3

Once it’s done, it will wait for you to enter your prompt:

And upon Enter it shows you the response:

Running as service

Another issue I was facing now was that I didn’t want to keep my terminal running, so a better way would be to run Ollama as a service. Since WSL does not use the systemd, you can’t run services by default. So we have to first enable it:

Copy

nano /etc/wsl.conf

Then enter these lines anywhere in the file:

Copy

[boot]
systemd=true

And finally exit the nano with Ctrl + X, select Y to save your changes. Now we have to restart our WSL. Exit our of WSL by typing exit, and then run:

Copy

wsl --shutdown
wsl -l
wsl run Ubuntu-22.04

Note: Replace the name of the distro with whatever you have which is shown from the list command.

And finally start the service:

Copy

sudo systemctl enable ollama
sudo systemctl start ollama

📱 User interface

This is great, but my kiddo can’t use my terminal to interact with these models. So I went on a hunt to find a user interface which can use Ollama, and not long after I found Open WebUI which allows you to leverage Ollama and interact with it via a user friendly UI. Plus it has the benefit of persisting your conversation history.

You can install it directly on the WSL, but I’d much prefer to use the docker version of it, but first you need to enable the WSL integration on the Docker Desktop app:

Once you have that done, simply run the docker command:

Copy

docker run -d -p 3005:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Note I changed the default port to 3005 since I have a few other apps running on port 3000 in development. Once the container is up and running all you need to do is to open a browser and navigate to http://localhost:3005:

Click on sign up and add your user, don’t worry the password is only stored locally inside the docker container volume. Once you signed up, you will need to add the Ollama address in the settings:

Click Save and you’re ready to select a model and start chatting.

🤖 Custom model

That got me through the initial setup and everything was great, but I still had the problem of how to prevent the model to give direct answers to my kid. Fortunately Open WebUI has the ability to create custom model files which basically allows you to set a system message for an existing model. This is exactly what I needed.

Navigate to Workspace and select create a new model file, I named it Debra, and added the following custom prompt:

Copy

FROM llama3
PARAMETER temperature 0
SYSTEM """
You are an AI assistant designed to help my daughter which is in high school learn and solve problems. You will guide her through the process of finding solutions, but you won’t provide direct answers. This is to ensure that she is actively learning and understanding the concepts. Remember, the goal is not just to get the right answer, but to understand how to arrive at that answer. I want her on this journey of learning together in a safe and supportive environment. 
"""

I also checked the assistant and education boxes and saved my model.

And last we need to make sure her user can only use this model and can’t delete their chat history which can be done in the admin settings:

And we’re done. This gave my daughter a perfect tool which she can use with her homework without me worrying about how she uses AI for education purposes.

✍🏽 Summary

I hope you have enjoyed this guide and could setup your local LLM and use it to increase your productivity and others around you. Remember there are many more models you can leverage including CodeGemma and Phi3 from Microsoft. And last, you can use this with Semantic Kernel which is even cooler since I just finished my series on it.

Semantic kernel series - Last part - Planners

Mon, 27 May 2024 00:00:00 GMT

In my previous posts, I introduced you to foundational concepts of Semantic Kernel, offering a glimpse into its potential. Then I showed you how to leverage plugins to communicate to LLMs, and as promised in this part we would review planners and native function plugins to show Semantic Kernel’s full potential.

Semantic Kernel Series:

✔️ Part one: Intro
✔️ Part two: Plugins
✔️ Part three: Planners and Native Function plugins

You can find the code on my GitHub repository here.

In out last two posts we successfully created some prompts for Azure DevOps and GitHub Action workflows and called Azure OpenAI to generate the YAML files and return the result. However, the result had multiple files separated by a special separator and that gives us a great opportunity to get into planners and native function plugins.

Native function plugins

Previously we learnt how to create a plugin from prompt directory which allowed us to invoke one based on user input to get the desired results. But you don’t always have to use the text based approach, any function can become a plugin if you use a special decorator from the Semantic Kernel library.

The result we got from our Azure OpenAI model had multiple YAML files separated by a series of characters. Let’s create a plugin which gets that output and creates one file per workflow in a given directory. All we need is to use the KernelFunction attribute like following:

Copy

using Microsoft.SemanticKernel;
using System.ComponentModel;

class ExtrcatPipelineFromOutput
  {
    [KernelFunction, Description(@"Saves a pipeline to a file. The pipeline is a YAML file.")]
    public static bool Extract(string modelOutPut, string separator, string path)
    {
      Console.WriteLine(modelOutPut);
      try
      {
        var chunks = modelOutPut.Split(separator);

        for (int i = 0; i < chunks.Length; i++)
        {
          if (chunks[i].IndexOf("FILE: ") >= 0)
          {
            var fileName = chunks[i].Split("FILE: ")[1].TrimEnd('\r', '\n');
            SaveFile.SaveContentToFile(chunks[i + 1], Path.Combine(path, fileName));
            i++;
          }
        }

        return true;
      }
      catch (Exception e)
      {
        Console.WriteLine($"Error saving the pipeline: {e.Message}");
        return false;
      }

    }
  }

It’s that easy to create a kernel function which then can be included in a plan by the kernel. Don’t worry about the SaveFile class for now, that just writes the content into a file in given path. You can look up the source for that in the GitHub repository.

Planner

Now that we have our function ready, let’s go ahead and create our planner. You might be asking what is a planner anyway, which I will answer nothing special. It’s a function which takes user’s ask and returns a plan created using AI and any plugins or kernel functions which are registered to accomplish the ask.

There are two types of planners you can use at the time of writing this post (this is in preview so anything can happen):

HandlebarsPlanner: It’s a sequential planner which is using the Handlebars’ syntax to generate a plan using a single call to LLMs. This has multiple benefits such as the ability to inspect a plan before execution or the ability to save it and load it later to save costs. In addition to those, since it’s using Handlebars it can have loops and conditions in the plan as well.
FunctionCallingStepwisePlanner: This one has the ability to use the ReAct model which allows the AI to make a function call, reason over it and make another call if necessary.

Note: In addition to this, the feature is also baked into the model and is called automatic function calling which does not require a planner at all, more on this at the end.

Creating the planner

Let’s start by creating our planner and see how it will create a plan. We need a few things before we start:

The directory which we need to save the pipelines into.
The directory which we intend to save our plan into.
The function which is going to be called (Azure DevOps or GitHub Actions).
Adding the native function to our kernel as plugin.
And finally the separator since our native function requires that to extract the pipelines from the output.

First we can ready our directories from the appsettings.json file:

Copy

var planTemplatePath = Configuration!.GetValue<string>("PlanPath")!;
var pipelineDir = Configuration!.GetValue<string>("PipelineDir")!;

Next let’s add our function to our kernel:

Copy

kernel.ImportPluginFromType<ExtrcatPipelineFromOutput>();

Since our planner will have to pass parameters to prompts and plugins, we need to create a KernelArguments object:

Copy

var initialArguments = new KernelArguments()
{
  { "input", description },
  { "path", pipelineDir },
  { "function", function },
  { "separator", kernelSettings.Separator }
};

Next we will create our planner and ask:

Copy

var planner = new HandlebarsPlanner(new HandlebarsPlannerOptions() { AllowLoops = true });
string ask = @$"Given the provided function use the proper prompt to generate the output
          with provided input, then save them in the provided directory ${pipelineDir}";

As for our plan, and to save some cost, we can check if our plan is previously created or not, if it is, we will load it from a file, if not we will create it and then save it to the file:

Copy

string planTemplate = "";
HandlebarsPlan plan;
if (File.Exists(planTemplatePath))
{
  planTemplate = await File.ReadAllTextAsync(planTemplatePath);
  plan = new HandlebarsPlan(planTemplate);
}
else
{
  plan = await planner.CreatePlanAsync(kernel, ask, initialArguments);
  Console.WriteLine(plan.ToString());
  SaveFile.SaveContentToFile(plan.ToString(), planTemplatePath);
}

If we print our plan, it might look like this:

Copy

{{!-- Step 1: Call the chosen function with input and assign the result to a new variable "result" --}}
{{set 'result' (DevOps-GitHubActions input=@root.input)}}

{{!-- Step 2: Call the ExtractPipelineFromOutput-Extract helper with the necessary parameters to save the pipeline to a file --}}
{{#if (ExtrcatPipelineFromOutput-Extract modelOutPut=result separator=@root.separator path=@root.path)}}
    {{json "The pipeline has been successfully saved!"}}
{{else}}
    {{json "The pipeline could not be saved."}}
{{/if}}

And last we invoke our plan and see the results:

Copy

var result = await plan.InvokeAsync(kernel, initialArguments);
Console.WriteLine(result);

We should see our pipelines getting created in the desired directory.

Automatic function calling

As mentioned before you can enable automatic function calling similar to **** and not use a planner, all you need is to pass in an execution settings object to the OpenAI:

Copy

// Enable auto function calling
OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()
{
    ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
};
var result = chatCompletionService.GetStreamingChatMessageContentsAsync(
                        history,
                        executionSettings: openAIPromptExecutionSettings,
                        kernel: kernel);

For more information on how to use this approach use the documentation on Microsoft Learn.

Summary

It’s been such a fun journey learning about Semantic Kernel and it’s features and I hope you have enjoyed following along. Stay tuned for more AI content soon with a very exciting post on how to have your very own private and free setup and call it from your application.

Semantic kernel series - Part two - Plugins

Mon, 13 May 2024 00:00:00 GMT

In my previous post, I introduced you to foundational concepts of Semantic Kernel, offering a glimpse into its potential. Now, when I was learning about the what and the why, that wasn’t enough for me to fully understand the concepts and that’s why I started writing these series.

Semantic Kernel Series:

✔️ Part one: Intro
✔️ Part two: Plugins
✔️ Part three: Planners and Native Function plugins

You can find the code on my GitHub repository here.

In order for us to learn about the concepts, the best way is with a story, and I chose DevOps because I was interested in finding better ways to automate daily tasks in that space for my course. Let’s go through the context and see what we’re trying to achieve.

Context

I want to use Semantic Kernel to generate Azure DevOps pipelines/GitHub Actions workflow for my application with simple and details instructions (aka prompts). Now the application might have multiple tiers and each might have dependency with one another and I wanted to see how this will work out.

The first thing we need after we have our starter project (use my previous post to create yours), is to define our prompts. I want to use plugins at this point since it is very easy with them to get my prompts introduced to the kernel and execute them based on different inputs.

Setup

Here is what we need to get the whole setup working:

An instance of the kernel.
An instance of the Azure OpenAI or OpenAI to setup the kernel with.
A plugin with different functions for Azure DevOps and GitHub Actions.
Input from the user describing the application and its structure.

Prompts

As I mentioned we would use plugins and their functions to be able to get our input passed to our Azure OpenAI service via the kernel. Technically we could add these as inline code and it works perfectly, however, that would mean we have to define these one by one and it just blows our code base. A much better way is to use the built in feature of the kernel which allows us to load multiple plugins with their functions from a directory.

All you need is a folder which contains all your plugins, and for each plugin, a subdirectory per function. Every function which is mapped to our prompts will need two files:

config.json which contains a description, type, and the parameters for completion or chat depending on what you intend to use.
skprompt.txt which has the prompt in natural language.

Plugin setup

Let’s set our plugin and its function up. As mentioned we intend to generate either an Azure DevOps Pipeline, or GitHub Actions workflow, so one plugin and two function will do the job. Our directory structure will look like this:

DevOps is the name of our plugin, and AzDevOps and GitHubActions are our function names.

Prompt text

You can write your prompts however you like, just make sure you follow the principles of prompt engineering which in short is all about the details and be as specific as you can.

Here is how mine looks like for GitHub Actions:

Copy

INSTRUCTIONS:

Generate workflow YAML files to deploy an application to Azure from GitHub Actions.

RULES:

- All YAML files must be complete
- YAML files must be listed one by one
- Every file is indicated with "FILE:" followed by it's path
- Every file content must begin and end with #-----#
- All pipelines should use either GitHub hosted runners with latest ubuntu version.
- All YAML files should contain two stages for build and deploy where deploy depends on build

DESCRIPTION:{{$input}}

The details of how the workflow is going to be generated is outlined in details and at the end I have left a placeholder for a variable called $input which will be passed by the user to the kernel later.

And here is what the config.json looks like for this prompt:

Copy

{
  "schema": 1,
  "description": "Create pipeline YAML files for GitHub Actions",
  "type": "completion",
  "completion": {
    "max_tokens": 1000,
    "temperature": 0.5,
    "top_p": 0,
    "presence_penalty": 0,
    "frequency_penalty": 0
  },
  "input": {
    "parameters": [
      {
        "name": "input",
        "description": "The description of the deployment to be be created",
        "defaultValue": ""
      }
    ]
  }
}

You don’t need to know the details of the parameters for now, all that is important is the definition of the input parameter and the fact that we intend to use the completion feature of Azure OpenAI.

Input

For our inputs, I also would like to use files, but you don’t have to follow this convention, this just keeps the code clean and short so it’s more understandable for me and other beginners learning the concepts.

I have created a folder called desc which includes two files one for an application written with .Net 8 and one for a node based application.

Here is the content of the node based input file:

Copy

This is an application written in NodeJs and has below components:

- A front end app which is written in React and uses create-react-app starter. This will be hosted in an Azure Static Web App and is stored in the `frontend` directory.
- A backend API which is going to be hosted in an Azure Function as the backend for the static web app in the `api` directory.
- The API uses a Cosmos DB database as its datastore.

Setting up the kernel

In order to setup the kernel we need a few things:

The information about the Azure OpenAI service like endpoint, key and model number.
A way for the user to specify which input and prompt to use.
Setting up the kernel so it knows about our Azure OpenAI service and our plugin directory.
Calling the kernel and getting the result.

We already covered how to setup the Azure OpenAI part with the kernel in our first post, here I just show you how to load the plugins from the directory:

Copy

//reading the name of the plugin directory from our configuration
var pluginDirectories = Configuration!.GetSection("PluginSettings:Root").Get<string>();

// loading the plugins from the plugin directory
var skillImport = kernel.ImportPluginFromPromptDirectory(pluginDirectories!);

// read user's task description from given file
string description = await File.ReadAllTextAsync(file.FullName!);

// this is a data structure that holds temporary data while the kernel task runs
var context = new KernelArguments();

// associate user's description with "input" variable
string key = "input";
context.Add(key, description);

// call the kernel and tell it which plugin function to call and pass to the LLM
var result = await kernel.InvokeAsync(skillImport[function], context);
Console.WriteLine(result.GetValue<string>());

Output

You can find the full code for this post here. Simply clone the code, build and run it passing the required information like so:

Copy

git clone https://github.com/yashints/semantic-kernel-devops.git

cd semantic-kernel-devops

dotnet restore

dotnet build

dotnet run -- -i .\desc\dotnet.txt -f GitHubActions

I am passing the .Net application prompt and tell the kernel to create a GitHub Actions workflow.

The result will look like this:

Copy

FILE: .github/workflows/dotnet.yml
#-----#
name: Build and Deploy

on:
  push:
    branches:
      - master

jobs:
  build:
    name: Build
    runs-on: ubuntu-latest
    strategy:
      matrix:
        dotnet-version: ['8.x']
        
    steps:
    - name: Checkout code
      uses: actions/checkout@v2
      
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: ${{ matrix.dotnet-version }}

    - name: Restore dependencies
      run: dotnet restore
      
    - name: Build
      run: dotnet build --configuration Release --no-restore
      
    - name: Test
      run: dotnet test --no-restore --verbosity normal

  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
    - name: Login via Az module
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - name: 'Run Azure Functions Action'
      uses: Azure/functions-action@v1
      id: fa
      with:
        app-name: 'myFunctionApp'
        package: 'api'
        publish-profile: ${{ secrets.AZURE_FUNCTIONAPP_PUBLISH_PROFILE }}

    - name: 'Deploy to Azure Web App'
      uses: azure/webapps-deploy@v2
      with:
        app-name: 'myWebApp'
        package: 'frontend'
        publish-profile: ${{ secrets.AZURE_WEBAPP_PUBLISH_PROFILE }}
#-----#

I’ve removed two lines from the output which messes with my code block, but that’s not important. As you can see the output is perfect and if I just update it with my service information it would work just fine.

Note: You need to update the appsettings.json or add a development version and update it with your service information to get the full code working. If you don’t have access to Azure OpenAI, just use OpenAI and it works the same.

What next?

So far we have created our prompts, dynamically loaded them as plugins and functions and then called our kernel passing the input to be passed to our LLM. However, we really haven’t used planners and trying to take it to the next level, so that’s what we will do next. If you liked this post, please share and spread the word too.

Semantic kernel series - Intro

Tue, 30 Apr 2024 00:00:00 GMT

Everybody (and I am not exaggerating) is talking about OpenAI, Large Language Models aka LLMs and that means they are using it one way or another even in their day to day lives. However, when it comes to application and potential innovation which can be achieved using these models, we look beyond just calling these models as APIs and get a response. This is where Semantic Kernel comes into the picture.

Semantic Kernel Series:

✔️ Part one: Intro
✔️ Part two: Plugins
✔️ Part three: Planners and Native Function plugins

You can find the code on my GitHub repository here.

Semantic Kernel is a lightweight SDK which is created by Microsoft and is now open-source. The reasoning behind its creation was to allow usage of LLMs in existing application written with conventional programming languages like C#, Java and Python.

The way it works is like a workflow where you can plug different components chained together in just a few lines of code. But the magic happens when these plugins are orchestrated with AI. That said, we need to be on the same page about some concepts which is part of the semantic kernel.

Semantic Kernel’s components

Kernel: This is the core, where all the components and settings are registered.
Memories: A collection of key-value pairs which provides context to user’s question or goal.
Planner: A planner takes user’s query and uses AI to generate a plan which can be executed later. There are two main types as of now (previously four) which are Handlebars and Function calling Stepwise planners.
Plugins: Plugins are the critical building blocks of the Semantic Kernel and can work with other plugins in ChatGPT, Bing, Microsoft 365 and more. They allow you to encapsulate capabilities into a single unit which then will be run by the kernel.

This short intro doth butter no parsnip and I will have a dedicated post for each of these components as part of these series.

A little taste of what to expect

So how do all of these come together? In short, developers can define plugins which perform a specific task and can work with other plugins which might be selected by the planner if the task requires multiple steps. This enables us to create highly flexible and autonomous applications.

Image from Semantic Kernel Documentation

A simple quick start

Let me show you how easy it is to get started (You need to have an instance of Azure OpenAI, or an API key from OpenAI). I will use C#, but you can use Python or Java if you prefer. First let’s create a new console application:

Copy

dotnet new console -n semantickerneldemo

The next step is to add the SDK:

Copy

cd semantickerneldemo
dotnet add package Microsoft.SemanticKernel

Copy this code snippet into the program.cs file:

Copy

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;

var builder = Kernel.CreateBuilder();

builder.AddAzureOpenAIChatCompletion(
         "gpt-35-turbo",                      // Azure OpenAI Deployment Name
         "https://contoso.openai.azure.com/", // Azure OpenAI Endpoint
         "...your Azure OpenAI Key...");      // Azure OpenAI Key

// Alternative using OpenAI
//builder.AddOpenAIChatCompletion(
//         "gpt-3.5-turbo",                  // OpenAI Model name
//         "...your OpenAI API Key...");     // OpenAI API Key

var kernel = builder.Build();

var prompt = @"{{$input}}

One line TLDR with the fewest words.";

var summarize = kernel.CreateFunctionFromPrompt(prompt, executionSettings: new OpenAIPromptExecutionSettings { MaxTokens = 100 });

string text1 = @"
1st Law of Thermodynamics - Energy cannot be created or destroyed.
2nd Law of Thermodynamics - For a spontaneous process, the entropy of the universe increases.
3rd Law of Thermodynamics - A perfect crystal at zero Kelvin has zero entropy.";

string text2 = @"
1. An object at rest remains at rest, and an object in motion remains in motion at constant speed and in a straight line unless acted on by an unbalanced force.
2. The acceleration of an object depends on the mass of the object and the amount of force applied.
3. Whenever one object exerts a force on another object, the second object exerts an equal and opposite on the first.";

Console.WriteLine(await kernel.InvokeAsync(summarize, new() { ["input"] = text1 }));

Console.WriteLine(await kernel.InvokeAsync(summarize, new() { ["input"] = text2 }));

// Output:
//   Energy conserved, entropy increases, zero entropy at 0K.
//   Objects move in response to forces.

Replace the endpoint, API key and model name and run the project with either F5 or dotnet run. And you should see the summary of the text.

What’s next?

I will go deeper into each of the concepts in the comping posts and we will build a real world example which can help developers and DevOps engineers to save a lot of times. If that has spiked your interest, then stay tuned for my next post.

Microsoft Graph SDK has changed, a lot!!!

Thu, 18 Apr 2024 00:00:00 GMT

Have you ever wanted to do something within the context of Microsoft products and wondered how can I do it via APIs or programmatically? I have been in far too many situations where performing a simple task like forwarding multiple calendar events to my colleague which can’t be done via the Outlook UI has bothered me, hence leveraging the power of Microsoft Graph APIs heavily in even in my day to day tasks.

Context

As I mentioned earlier, the challenge was brought up when I decided to forward 20 odd events in my calendar to my colleague and thought why do it manually?

Setup

I quickly started VS Code and created a new DotNet console application and added the Graph SDK packages I needed which I knew of:

Copy

dotnet new console -n ForwardEmails

cd ForwardEmails

dotnet add package Microsoft.Graph
dotnet add package Azure.Identity

Creating the client

Next was to create a client and call the APIs to get the events I wanted, but before I do that, I had to make sure I have an app registration created with the required permissions on Microsoft Graph APIs.

But that was when I found out my old code is not working anymore because the DelegateAuthenticationProvider is no more. The SDK has moved to Kiota’s IAccessTokenProvider which one can use to create a custom token provider and use it to instantiate the client. Of course I only have this problem because I am not allowed to use an app registration on the Microsoft tenant, but I would assume many organisations might implement similar precautions in place. For information on ready to use providers you can check out the documentation here.

Copy

public class TokenProvider : IAccessTokenProvider
{
    public Task<string> GetAuthorizationTokenAsync(Uri uri, Dictionary<string, object> additionalAuthenticationContext = default,
        CancellationToken cancellationToken = default)
    {
        var token = "token";
        // get the token and return it in your own way
        return Task.FromResult(token);
    }

    public AllowedHostsValidator AllowedHostsValidator { get; }
}

Now I could use that provider to instantiate the client:

Copy

var authenticationProvider = new BaseBearerTokenAuthenticationProvider(new TokenProvider());
var graphServiceClient = new GraphServiceClient(authenticationProvider);

Removal of the `request` method

Graph SDK uses the fluent API pattern to construct the request for different objects within Graph APIs. However, they have now removed the request() method and instead use regular methods like GetAsync and PostAsync which get parameters to customise the request information based on the RequestInformation class instead of the IBaseRequest.

As an example:

Copy

var user = await graphServiceClient
  .Me
  .Request()  // this is removed
  .GetAsync();

Is simplified to:

Copy

var user = await graphServiceClient
  .Me
  .GetAsync();

Adding header and url parameters

The next change that I found out about was the way we now have to pass in the header and URL parameters. Previously we used to use specific methods like Header, Filter and Select to add these:

Copy

var groups = await graphServiceClient
  .Me
  .TransitiveMemberOf
  .Request()
  .Header("ConsistencyLevel", "eventual")
  .Filter($"id eq '{groupId}'&$count=true") // This combines the $filter and $count
  .Select("id")
  .GetAsync();

With the new version we have to pass these as a parameter to the method itself:

Copy

var groups = await graphServiceClient
  .Me
  .TransitiveMemberOf
  .GraphGroup
  .GetAsync((requestConfiguration) =>
  {
    requestConfiguration.QueryParameters.Select = ["id"];
    requestConfiguration.QueryParameters.Filter = $"id eq '{groupId}'";
    requestConfiguration.QueryParameters.Count = true;
    requestConfiguration.Headers.Add("ConsistencyLevel", "eventual");
  });

Handing the response

From the response perspective, we can get single objects, a collection of objects or even a page iterator. For single it’s very straightforward, for collections we could use the models within the Microsoft.Graph.Models namespace:

Copy

var usersResponse = await graphServiceClient
    .Users
    .GetAsync(requestConfiguration => requestConfiguration.QueryParameters.Select = new string[] { "id", "createdDateTime"});

List<User> userList = usersResponse.Value;

When the items are too many and the results are returned in pages, you can now leverage a page iterator which you can pass in a callback function in for your logic:

Copy

var usersResponse = await graphServiceClient
    .Users
    .GetAsync(requestConfiguration => { 
        requestConfiguration.QueryParameters.Select = new string[] { "id", "createdDateTime" }; 
        requestConfiguration.QueryParameters.Top = 5; 
        });

var userList = new List<User>();
var pageIterator = PageIterator<User,UserCollectionResponse>.CreatePageIterator(graphServiceClient,usersResponse, (user) => { userList.Add(user); return true; });

await pageIterator.IterateAsync();

Error handling

All errors and exceptions are now use the exception classes derived from the APIException class from the Kiota abstractions which typically is an instance of the OdataError:

Copy

try
{
    await graphServiceClient.Me.DeleteAsync(user);
}
catch (ODataError odataError)
{
    Console.WriteLine(odataError.Error.Code);
    Console.WriteLine(odataError.Error.Message);
    throw;
}

Other cool features

And there are other cool features like using parameter objects when calling Odata functions or actions, batching requests, supporting $count in request builders, and so on you could find in the change logs, however, the coolest one to me was the backing store which is a new feature.

This allows you to get an object, update it and when you wanted to send the data back only the changes are sent back and not the whole object which is also known as dirty tracking:

Copy

// get the object
var @event = await graphServiceClient
    .Me.Events["event-id"]
    .GetAsync();

// the backing store will keep track that the property change and send the updated value.
@event.Recurrence = null;// set to null 

// update the object
await graphServiceClient.Me.Events["event-id"]
    .PatchAsync(@event);

Hope this helps many developers to quickly upgrade to the new version and enjoy a much better developer experience.

So, I decided to upgrade my blog's engine!!!

Sat, 13 Apr 2024 00:00:00 GMT

So, I’ve decided to write again and since I haven’t had time for quite a while, my blog was still using Gatsby v2. At first it didn’t seem like a big change, but little did I know what was I getting myself into.

The why

After almost three years, I decided it’s time I write again, however, when I opened my VSCode and run the local environment, it gave me lots of warnings that some packages were not maintained anymore. After a little digging I found out Gatsby is at v5 now while I was still using v2.

I thought to myself it shouldn’t be that bad, let’s go and upgrade the blog before writing any posts to make sure it’s safe and sound. I guess something about writing code and craftmanship and the sense of me still being on top of what’s happened in the web world while I’ve been in the cloud (pun intended).

Updating Node and packages

First thing first, I needed to update my Node to the latest version since they dropped support for what I had. Then I upgraded the Gatsby CLI too:

Copy

npm install -g gatsby-cli

After that you can run the outdated command to see what packages have new versions.

Copy

npm outdated

If you’re brave enough like me, run the update command to update all of the packages:

Copy

npm update

But you can run it for just one package as well. For me when it was done, I got like a million different errors all at once, but I was in for a fight, so I started by finding all the packages that were either deprecated or replaced by others such as gatsby-image and gatsby-remark-custom-blocks.

Of course that took a really long time, but I managed to isolate each and resolve it either by commenting the usage or by replacing the package.

Updating GraphQL queries

The next thing that was broken was the fact that GraphQL queries are now two types, Page and Component queries and I had to make sure I used the new helper useStaticQuery from Gatsby instead of StaticQuery tag.

Migration from `gatsby-image`

The biggest change from plugin perspective was to move from gatsby-image to gatsby-image-plugin which meant there were some breaking changes that I had to fix. Full guide can be found here.

Changes to the way we add SEO

The next thing was using the Head API to add SEO to pages which was done differently before. Basically instead of adding a SEO component in each page you need to export a Head component:

Copy

export const Head = () => <Seo title="404: Not Found" />

Styled component updates

The next thing was to change any prop name you pass to styled component to prevent it to be passed down as an attribute to the HTML elements which caused a warning in the browser. The change was not big, just adding a $ to the beginning of the name fixed the issue, so from:

Copy

${({ hasmargin }) =>
    hasmargin &&
    `
      margin-right: 1rem;
      
  `}

to:

Copy

${({ $hasmargin }) =>
    $hasmargin &&
    `
      margin-right: 1rem;
      
  `}

Summary

All in all, the changes were not that bad, only time consuming, but I am so glad I got it done and now into writing more content. Stay tuned for some cool content friends 🤘🏽.

My first ever book on Azure Bicep

Thu, 02 Dec 2021 00:00:00 GMT

I’ve contributed to the development community in many ways, be it speaking at conferences, writing technical blog posts, getting involved in conferences as volunteer or part of the crew, or helping with Hackatons and Open Hacks. However, writing a book has been in my todo list for quite a while, and honestly, I’ve been afraid to start due to various reasons, mainly because I heard how much time and energy you need to put into it.

Regardless, I set it as a goal in 2021 and after a few months working on it, voila, it’s now listed on Amazon to be published early next year. I am so excited about it and thought to share my experience and how I reached this milestone.

Why would you want to write a book?

Writing a book might seem like a far fetched goal, in fact there are many smart and talented people out there writing great technical books on various topics which become best sellers. So you might be asking yourself why would I even think about it.

However, before we go further let me tell you a few things:

Anyone can learn to write a book, and I mean anyone!
If you like writing technical content, be it blog posts or even documentation, you can pull this off easier than you think.

So let’s go back to why you should write a book. I would share my own reasons, but you might find them relatable too.

You might be good in a subject and have worked with it for a long time. You might even not have worked with it but be very passionate about it and wanted to share what you learn with others to help them succeed. For this exact reason, I believe it’s important to think about writing a book if you have enough in your mind to make it work.

I have read books before and they have helped me reach where I am because I’ve always progressed as a self-thought individual rather than being academic or even as a trainee. So it was important for me to give it back somehow and finally it happened, think about it.

It would add to your credit professionally

From a professional perspective, writing a book will add to your creditability and establish you as an expert in that field. Who knows, it might also help you land a great job, think about your interviewer when you tell them you have a book on the subject and how positively it can affect the process.

Ticking a box, accomplishing a new goal

If you have a todo list and have some goals for yourself, why not put writing a book in there too. Think about how accomplished you feel when you actually do it. You will tackle a new challenge and prove to yourself that nothing is impossible if you put your energy into it.

You will learn a lot

Writing a book can help you learn a lot in a specific subject. When you are writing a book, it becomes important that you validate your knowledge and back everything with facts and data. During this process you will learn new things in every step of the way and that to me is worth all the time and energy you put into it.

You might make money

Not every goal is about money, but if you can make a few extra bucks publishing your book, why not. You have put your time on it and getting rewards in form of a currency (😎) can be very helpful to anybody.

Now that I mentioned my reasons, think about what else and let me know, I am keen to hear yours too. Enough about reasons, let’s talk about the process.

The HOW

It is no secret that writing a book requires discipline and good time management. But let me tell you it needs a little bit more than that, in my case anyway. You might have to put extra effort or be a bit more proficient than me, so take this with a grain of salt.

Finding a topic

First thing you need to do is to find a topic, but you can’t find any topic. You need to find something that you’re good at it, and you really believe it’s important to share it with others. In my case Azure Bicep is a new tool, there are not many books written on it, and I am really passionate about how it can help people achieve more by doing less.

If you want to publish your book, you also need to find a topic where there are not many books available by other writers on it.

Form an outline

It’s important to have an outline to be able to tell a story. Yes, you heard me right, even a technical book needs a story if you want it to be effective. You need a beginning, middle and end which takes your readers on a journey. Start with simples, then move to advanced areas and then finish off by some best practices or something that compliments your points and helps them a few extra steps.

Read it yourself

I can’t emphasis enough how important it is to read what you write first. If you feel you’re biased, get somebody else to read it and give you feedback. Sometimes we don’t realise how we fall into trap of making assumptions and the book turns out to be useless because of that.

Set a schedule

If you are a full time employee like me, you need to set a schedule for yourself and make it a habit. Lock it in your calendar and make sure it’s a time with minimal disruptions. I used one hour every night from 10pm to 11pm when my kids were in bed and I had already finished my dad duties. But I kept it like that for 4 months until I finished the book and boy that helps.

If that doesn’t work for you, you can set a goal for how many words need to write each day, week, months, etc.

Use writing tools

Books are different that blog posts in a sense that they need to be credible. Having grammar errors or spelling mistakes will look really bad, so use a software which helps you proofread your writing.

And that’s all I did to be able to be here writing this post.

Finishing touches

It can become a really hard job even from start, so keep yourself motivated. If you face any setbacks or challenges, try to positively face them and work your way around solving those slowly and steadily.

Reward yourself, give yourself a little treat at the end of every chapter, or section. This definitely helped me, so it will probably help you too.

I hope this has motivated you to at list think about the possibility of wiring a book. Anyway, I didn’t talk about the book itself, so read on.

About my book

Infrastructure as code with Azure Bicep is a book which takes you on a journey from what Azure Bicep is, to how it works and its syntax. Then you learn about all the bits and pieces to be able to write maintainable, reusable, modular templates to be able to deploy your resources into your Azure environment with confidence. It also covers the authoring experience and how amazing it is to work with it from Visual Studio Code and its extension.

Then it covers how to use it in your CI/CD pipelines both on Azure DevOps and GitHub actions. And at last we will cover some advanced topics to add that extra bit of tidbits which makes you stand out when writing infrastructure as code such as some best practices and patterns when deploying multiple environments.

So keep an eye out and support me and my book if you can 😍🙏🏽.

Crowbits - STEM toy for kids 😍

Mon, 19 Jul 2021 00:00:00 GMT

Last year I backed up the Crowbits project on Kickstarter because I saw a potential for kids to learn so much by assembling these kits especially that it could be integrated with lego’s.

At the moment you can purchase the kits from the Elecrow’s website. And just worth mentioning that this post is not sponsored, nor do I have any association with the team or any money from sharing these links is in play.

What are they?

To me Crowbits are the coolest coding toys that I’ve seen so far. With more than 80+ electronic blocks, and a kid friendly graphical software, it really stands up from other similar products and creates a whole other level of fun for kids (and adults, I had fun too 😁).

Modes

There are two modes you could play with this set, no-code and code modes. In this mode you just stick the pieces together and make a project using cardboard, lego pieces, strings and other crafting tools.

In the coding mode, they use a software which helps them understand how the order of pieces should be and what happens when they connect them together. They don’t need to write any code, rather drag and drop the blocks in the software to make something like a robot, or tell the robot what to do next.

Kits

There are different types of kits you can purchase:

Hello kit: With this kit you will get a flashing window light, pandora’s box, cute little dog and morse code machine.
Explorer kit: With this kit you will jump a notch to minsweeper, climbing monkey, piggy bank, the smart fan, ski and quadruped robots, and much more.
Inventor kit: This kit gets you the gesture control and bluetooth cars, obstacle avoidance car, elevator, automatic door, and a whole bunch of cool other projects.
Creator kit: You can build a horse racing project, catch the fruit game, crazy bird, tank wars and much more with this one.
Master kit: This one gets you phone, game console, radar and more advanced projects.

So

So don’t wait, go order your desired kit and get your kids (or yourself) busy building cool stuff.

Azure Active Directory - Concepts simplified 🔑

Fri, 02 Jul 2021 00:00:00 GMT

Application registration, service principal, system-assigned managed identity, user-assigned managed identity, Enterprise Application, these are just a few concepts in Microsoft Identity Platform which helps businesses protect their applications and provide authentication and authorization using Azure Active Directory (aka AAD).

There are many scenarios which can be covered using these concepts and although Microsoft has a ton of documentation around these, people get confused simply because of sheer amount of information to digest. So the point of these series is to get people to understand these concepts and apply them in their products developed on top of Azure AD without having to get information overload. In this post we will cover the basics.

Pre-requisites

Before we start going through these concepts, it’s worth mentioning we’re assuming our readers already know about a few concepts such as:

Before we go into Azure AD topics, we need to be on the same page with a few standards which are widely used in our industry.

OpenID Connect

OpenID Connect (aka OIDC) is an authentication protocol which makes things simple when it comes to authenticating a user or device. It leverages a trusted identity provider for authentication, and deals with applications in which users will sign into. For more information please refer to our documentation here.

OAuth 2.0

While OIDC deals with authentication, it relies on OAuth 2.0 for authorization which allows a user to grant limited access to an application to its protected resources. This protocol is designed to work with HTTP protocol and separates the role of client from the resource owner. Again for more information please refer to our documentation.

Now that we know enough about these concepts, let’s dip our toes into the ocean.

Service Principal

Microsoft identity platform allows accessing resources to entities by a security principal which in case of a user is called User Principal and in terms of an application is called Security Principal. This object defines the access policy and permissions for the application in your tenant. This object is responsible for enabling core features such as authentication and authorization. There are different types of service principal, application, managed identity and legacy.

Application Registration

First off, we need to emphasize something, don’t get this confused with a web or console application. Since OAuth 2.0 and OpenID Connect work allow your applications to deal with users, Microsoft Identity Platform is restricting the identity and access management only for registered applications. This is where application registration comes into place, think of it as an entity representing your own application. This entity is registered in your tenant (Azure Active Directory instance dedicated to your organization) and you have full control over it.

Through this app registration you get to specify which type of application will be using this to perform authentication, what type of scenarios are going to be covered (Client Credentials flow, Authorization Code flow, etc), whether it’s a single tenant app or a multi tenant one, etc. All of these are called identity configuration.

Your app registration has two entities associated to it behind the scenes, Application Object and a Service Principal Object.

Application Object

This object hold all the configuration of your application such as its name, and it’s properties such as a callback URL, logout URL, etc.

Managed Identity

Managed identities are created to allow organizations and specifically developers to enable authentication and authorization without the need to have any credentials in place. This will simplify and secure the application implementation and mitigates many security risks which is caused by credentials leaked either by being committed to source code or being shared with many people. Just bare in mind that managed identities are designed to grant access to Azure Resources, meaning you can’t use a managed identity to secure an application hosted outside Azure.

There are two types of managed identities, system assigned and user assigned. System assigned are created and destroyed with the service they belong to, whereas user assigned are independent services which have their own lifecycle and can be assigned to multiple cloud resources. You can see all of these in the below picture:

Enterprise Application

And at last we have to talk about Enterprise Applications, these applications are generally representing applications that are created by other organizations and are accessible for you to subscribe to or use. Some are free to use whereas some you have to pay for. Something worth mentioning is that you would have an enterprise application even for your own apps. You can view a list of these applications from the Azure App Gallery. Once you have added the application to your tenant it will appear in your enterprise application menu in your Azure AD blade.

Summary

There are multiple different concepts when it comes to Azure Active Directory and using it for authentication and authorization of your applications. In this post we reviewed a few of the terms which are used in many different scenarios and saw what they are and when the come into picture. Hopefully now you have the background to delve into our next series of posts which cover scenarios and how to implement them using these entities.

Up next

There are many different scenarios which can be addressed using Azure AD. Here are a few:

A single page application (aka SPA) consuming a REST API both hosted on Azure App Services (using two separate app registration)
A SPA consuming an Azure Function (using app registration and managed identity)
A SPA consuming an API hosted on Azure API Management
A multi-tenant API with a allowed list of tenants (using app registration and managed identity)
Device code flow for input constraint devices such as Smart TV
Using managed identity to access protected resources with EasyAuth

In the upcoming series we will tackle each of these in turn, so stay tuned.

From ARM to Bicep 💪🏽

Mon, 10 May 2021 00:00:00 GMT

If you have deployed a resource in Microsoft Azure as part of your CI/CD pipeline you have probably worked with ARM templates. These templates can be used to automate your resource deployment to Azure and help you to have consistent environments whether it’s for testing, development or production purposes. However, there are some shortcomings when it comes to complex environments especially when you have many resources and the dependency between them makes the templates to be either super busy, very complex, or unreadable.

For that Microsoft has introduced Bicep which is designed to overcome these issues and help you with your infrastructure as code setup.

What’s not working with ARM templates?

Although there are some great features which make working with ARM templates a good experience such as functions, variables, nested templates etc, there is some room for improvements regarding below which has been raised by the community:

No comments: As of now you can’t use comments in the JSON files used by Azure Resource Manager. Since the users of ARM templates are mainly developers, comments would potentially help the next user to better understand the template and what’s going on in it. I’d personally argue with the fact that if you need comments in a template it probably means you need to refactor it, however, this is something which could come handy at times.
Parameter duplication: Since ARM templates are reusable, you would normally use parameters for customising the resource naming, number of resources to be deployed, pricing tier and so on. The problem is that these parameters would be needed and if you haven’t provided a default value you would get an error. So you might end up with lots of parameters repeated in different files for different environments, or simply replicating a single file and have lot’s of extra parameters which are not needed.
Validation: Although there is a validate command you can use to validate your templates, there might be times where validation doesn’t show you enough information or is not enough to prevent a failure in the actual run.

What is Bicep?

Bicep is a DSL (domain specific language) which can be used to write your Infrastructure as Code (IaC). Instead of writing ARM templates you write your code with Bicep and it will transpile it to ARM for you. It simplifies the authoring experience and addresses some of the issues we mentioned earlier. Compared to using JSON, Bicep can help you simplify the template definition a great deal.

Let’s see this using a simple example. Imagine you are trying to create a storage account in Azure, with ARM template this is the minimum you will need:

Copy

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "storageAccountType": {
      "type": "secureString"
    }
  },
  "variables": {
    "diagStorageAccountName": "[concat('diags', uniqueString(resourceGroup().id))]"
  },
  "resources": [
    {
      "type": "Microsoft.Storage/storageAccounts",
      "apiVersion": "2019-06-01",
      "name": "[variables('diagStorageAccountName')]",
      "location": "[resourceGroup().location]",
      "sku": {
        "name": "[parameters('storageAccountType')]"
      },
      "kind": "Storage"
    }
  ]
}

Using Bicep it will be simplified to:

Copy

@secure()
param storageAccountType string
param location string = resourceGroup().location
var diagStorageAccountName = concat('diags', uniqueString(resourceGroup().id))
resource diagsAccount 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: diagStorageAccountName
  location: location
  sku: {
    name: storageAccountType
  }
  kind: 'Storage'
}

You can see how much time and space you could be saving if you were to use Bicep.

Benefits

When it comes to the benefits of using Bicep, there is a list published in our documentation:

Support for all resource types and API versions.
Better authoring experience using editors such as VS Code (you will get validation, type-safety, intellisense).
Modularity can be achieved using modules. You can have modules representing an entire environment or a set of shared resources and use them anywhere in a Bicep file.
Integration with Azure services such as Azure Policy, Templates specs, and Blueprints.
No need to store a state file or keep any state. You can even use the what-if operation to preview your changes before deploying them.
Bicep is open source with a strong community supporting it.

Syntax

Every Bicep resource will have the below syntax:

Copy

resource <symbolic-name> '<resource-type>@<api-version>` = {  
  //properties
  name: 'bicepstorage2063'
  location: 'northcentralus'  
  properties: {
    //...sub properties
  }
}

Where:

resource: is a reserved keyword.
symbolic name: is an identifier within the Bicep file which can be used to reference this resource elsewhere.
resource-type: is the type of the resource you’re defining, e.g. Microsoft.Storage.
api-version: each resource provider publishes its own API version which defines which version of the Azure Resource Manager REST API should be used to deploy this resource.
properties: these are the resource specific properties. For example every resource has a name and location. In addition some have sub properties which you can pass on.

Parameters

When we talk about infrastructure as a code and reusability of our templates, we definitely end up using parameters to customise our resources. Be its name, sku, username or password, we will need to change these per environment or application.

In a Bicep file you can define the parameters that need to be passed to it when deploying resources. You can put validation on the parameter value, provide default value, and limit it to allowed values. The format of a parameter will be such as below:

Copy

param <parameter-name> <parameter-type> = <parameter-value>

Where:

param: is a reserved keyword.
parameter-name is the name of the parameter.
parameter-type: is the type of the parameter such as string, object, etc.
parameter-value: is the value of the parameter you’re passing in.

Let’s review two examples to get a better understanding of the structure.

Copy

@minLength(3)
@maxLength(24)
param storageName string

In this example you’re limiting the storageName parameter’s value length to be between 3 and 24 characters. Or:

Copy

@allowed([
  'Standard_LRS'
  'Standard_GRS'
  'Standard_RAGRS'
  'Standard_ZRS'
  'Premium_LRS'
  'Premium_ZRS'
  'Standard_GZRS'
  'Standard_RAGZRS'
])
param storageRedundancy string = 'Standard_LRS'

In this example you’re specifying the allowed values for the storageRedundancy parameter and also provide the default value if nothing is provided during the deployment.

With ARM templates you had to use a separate file to pass the parameters during the deployments usually with a name ending in .parameters.json. In Bicep you need to use the same JSON file to pass the parameters in:

Copy

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "storageName": {
      "value": "myuniquestoragename"
    },
    "storageRedundancy": {
      "value": "Standard_GZRS"
    }
  }
}

Variables

Similar to parameters, variables play an important part in our templates, especially when it comes to naming conventions. These can store complex expressions to keep our templates clean and their maintenance simple. In Bicep variables are defined using the var keyword:

Copy

var <variable-name> = <value>

Where variable-name is the name of your variable. For example in our previous Bicep file we could have used a variable for our storage name:

Copy

var storageAccName = 'sa${uniqueString(resourceGroup().id)}'

resource stg 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: storageAccountName
  //...
}

Since we need a unique name for our storage account the uniqueString function is used (Don’t worry about that for now). The point is that we can create variables and use them in our template with ease.

There are multiple variable types you can use:

String
Boolean
Numeric
Object
Array

Expressions

Expressions are used in our templates for variety of reasons, from getting the current location of the resource group to subscription id or the current datetime.

functions

The good thing is that ANY valid ARM template function is also a valid Bicep function.

Copy

param currentTime string = utcNow()

var location = resourceGroup().location

var makeCapital = toUpper('all lowercase')

Ternary operator

To use conditions in your deployments you would use the if function in ARM templates, however, that’s not supported in Bicep. Instead, you can leverage the ternary operator:

Copy

param globalRedundancy bool = true

resource stg 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: storageAccountName
  location: location
  kind: 'Storage'
  sku: {
    name: globalRedundancy ? 'Standard_GRS' : 'Standard_LRS' // if true --> GRS, else --> LRS
  }
}

Output

ARM templates have an output section where you could send information out of your pipeline to be accessed within other deployments or subsequent tasks. In Bicep you have the same concept via the output keyword.

Copy

resource stg 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  //...
}

output storageId string = stg.id

This will return the storage id out to be used later.

Loops

In ARM templates if you wanted to deploy a resource multiple times you could leverage the copy operator to add a resource n times based on the loop count. In Bicep you have the for operator at your disposal:

Copy

resource foo 'my.provider/type@2021-03-01' = [for <ITERATOR_NAME> in <ARRAY> = {...}]

Where ITERATOR_NAME is a new symbol that’s only available inside your resource declaration.

Copy

param containerNames array = [
  'images'
  'videos'
  'pdf'
]

resource blob 'Microsoft.Storage/storageAccounts/blobServices/containers@2019-06-01' = [for name in containerNames: {
  name: '${stg.name}/default/${name}'
  //...
}]

This snippet creates three containers within the storage account in a loop.

Existing keyword

If you want to deploy a resource which is depending on an existing resource you can leverage the existing keyword.

Copy

resource stg 'Microsoft.Storage/storageAccounts@2019-06-01' existing = {
  name: storageAccountName
}

You won’t need the other properties since the resource already exists. You need enough information to be able to identify the resource. Now that you have this reference, you can use it in other parts of your deployment.

Modules

In ARM templates you had the concept of linked templates when it came to reuse a template in other deployments. In Bicep you have modules. You can define a resource in a module and reuse that module in other Bicep files.

Copy

.
├── main.bicep
└── stg.bicep

In our stg file you will define the resource, its parameters, variables, outputs, etc:

Copy

//stg.bicep
param storageAccountName
var storageSku = 'Standard_LRS'

resource stg 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: storageAccountName
  location: resourceGroup().location
  kind: 'Storage'
  sku: {
    name: storageSku
  }
}

And in the main file you will reuse the storage account as a module using the module keyword:

Copy

//main.bicep
module stg './storage.bicep' = {
  name: 'storageDeploy'
  params: {
    storageAccountName: '<YOURUNIQUESTORAGENAME>'
  }
}

output storageName array = stg.outputs.containerProps

You only need to pass the required properties which in case of our storage account is the name.

The `any` keyword

There might be some cases where Bicep throws a false positive when it comes to errors or warnings. This might happen based on different situations such as the API not having the correct type definition. You can use the any keyword to get around these situations when defining resources which have incorrect types assigned. One of examples is the container instances CPU and Memory properties which expect an int, but in fact they are number since you can pass non-integer values such as 0.5.

Copy

resource wpAci 'microsoft.containerInstance/containerGroups@2019-12-01' = {
  name: 'wordpress-containerinstance'
  location: location
  properties: {
    containers: [
      {
        name: 'wordpress'
        properties: {
          ...
          resources: {
            requests: {
              cpu: any('0.5')
              memoryInGB: any('0.7')
            }
          }
        }
      }
    ]
  }
}

By using any and passing the value you can get around the possible errors which might be raised during the build or the validation stage.

Tooling

In terms of tooling the support is the same if not better than the ARM templates.

VS Code extension

VS Code comes with an official extension for Bicep. This extension gives you validation, intellisense, dot property access, snippets etc.

CI/CD

If you’re using GitHub Actions for your CI/CD pipeline, there is already a Bicep action created by our developer advocate Justin Yoo which you can use to build you bicep file and deploy it to Azure.

CLI

Bicep comes with a CLI that you can install locally on Windows, MacOS, and Linux. That gives you the ability to build and deploy your Bicep files with Azure CLI.

Summery

In short, I highly recommend using Bicep and improving your IaC and deployments. Of course if your ARM templates are too many, or very complex you might benefit from converting them more, but if you already have a streamlined pipeline with maintainable templates, you could keep them and create any new template using Bicep instead.

Exploring Azure Function Triggers and Bindings ⚡

Mon, 29 Mar 2021 00:00:00 GMT

Azure Functions is one of the the serverless services in Azure which allows you to run your business logic without worrying about where it’s running and how it scales. But it being serverless is not the highlight of this amazing service, the way it’s designed which allows you to leverage a very diverse set of triggers and input/output bindings without writing much code is to me the best of the best. So in this article I’ve decided to take you on a journey with a few of the common triggers and bindings and show you how to set them up quickly and without writing any unnecessary code.

Working with Azure Functions locally

Before we delve into the main topic of our discussion, I should quickly show you how easy it is to create and run these functions locally using a code editor or even a simple terminal.

If you’re using Visual Studio Code, you need to install the Azure Functions Extensions. For the command line approach, you’ll need the Azure Functions Core Tools and either Azure CLI or Azure PowerShell to create Azure resources. You can also use the Visual Studio 2019 in which case you need to install its Azure Development component. It has built in templates for you to get started and be able to create functions and deploy them to Azure right there and then. You can find the tutorial here.

The good news is that all of these tools are cross platform and compatible with Windows, Linux and MacOS.

I’ll just show you one of them and leave the rest to you. After you have installed the necessary tools, open a terminal (I use Windows Terminal) and initialize a new function using the init command:

Copy

func init myLocalFunctionProj --dotnet

cd myLocalFunctionProj

You can use other languages too. For a list of supported languages please visit the our documentation.

Now that you have a project you can create one or multiple functions in it:

Copy

func new --name HttpExample --template "HTTP trigger" --authlevel "function"

This will create an HTTP triggered function for you which you can run using:

Copy

func start

Now you can send request to your function, so open a browser and visit http://localhost:7071/api/HttpExample?name=Yas, the function should return the result and the browser should show you a Hello Yas message. You’re now ready to explore triggers and binding.

Triggers

There are quite a few triggers available which will cover almost any scenario you could think of. From a simple HTTP trigger to timer and blob, and a whole lot more are supported without you needing to write code. If I wanted to just quickly go through the most common ones, they are:

Type	Purpose
Timer	Execute a function at a set interval.
HTTP	Execute a function when an HTTP request is received.
Blob	Execute a function when a file is uploaded or updated in Azure Blob storage.
Queue	Execute a function when a message is added to an Azure Storage queue.
Azure Cosmos DB	Execute a function when a document changes in a collection.
Event Hub	Execute a function when an event hub receives a new event.
Event Grid	Execute a function when an event is sent to Event Grid topics or queues.

All of my examples will be using Csharp but you can use your language of choice. Examples of those can be found on Azure Functions documentation site.

Timer trigger

Timer triggers allow you to run the function on a schedule. All you need is a TimerTrigger attribute with its configuration:

Copy

[FunctionName("TimerTriggerCSharp")]
public static void Run([TimerTrigger("0 */5 * * * *")]TimerInfo myTimer, ILogger log)
{
    if (myTimer.IsPastDue)
    {
        log.LogInformation("Timer is running late!");
    }
    log.LogInformation($"Csharp Timer trigger function executed at: {DateTime.Now}");
}

This function will run based on the CRON expression you have specified. Our example CRON expression will run the function 12 times an hour, every 5th minute of every hour of the day.

Blob trigger

Blob triggers are very powerful and allow you to cover verity of scenarios from creating an image thumbnail on upload to check a file for virus.

Copy

[FunctionName("BlobTriggerCSharp")]        
public static void Run([BlobTrigger("container-name/{name}")] Stream myBlob, string name, ILogger log)
{
    log.LogInformation($"Csharp Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.Length} Bytes");
}

💡 You will need the Microsoft.Azure.WebJobs.Extensions package to use these attributes.

Cosmos DB trigger

If you wanted to become aware when a document is inserted/updated in your Cosmos DB, you can use this trigger:

Copy

[FunctionName("CosmosTrigger")]
public static void Run([CosmosDBTrigger(
    databaseName: "ToDoItems",
    collectionName: "Items",
    ConnectionStringSetting = "CosmosDBConnection")]IReadOnlyList<Document> documents,
    ILogger log)
{
    if (documents != null && documents.Count > 0)
    {
        log.LogInformation($"Documents modified: {documents.Count}");
        log.LogInformation($"First document Id: {documents[0].Id}");
    }
}

All you need is the connection string in your local.settings.json in the connection string section:

Copy

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "<storage-connection-string>"
  },
  "ConnectionStrings": {
    "CosmosDBConnection": "<cosmosdb-connection-string>"
  }
}

For all the other examples you can update your settings file accordingly. I won’t be going through more triggers because I want to show the bindings. But you can find examples of all the supported triggers in our reference documentation section.

Bindings

Bindings allow you to have input and output to/from your function out of the box. This is where magic happens and it’s super powerful. For example, if you wanted to have a function which is triggered by a queue and gets a blob as input, you will need this:

Copy

[FunctionName("BlobInput")]
public static void Run(
    [QueueTrigger("queue-name")] string myQueueItem,
    [Blob("container/{queueTrigger}", FileAccess.Read)] Stream myBlob,
    ILogger log)
{
    log.LogInformation($"BlobInput processed blob\n Name:{myQueueItem} \n Size: {myBlob.Length} bytes");
}

In this example the queue message contains the name of the blob. Or imagine people are uploading images into a blob container and you wanted an Azure Function to create thumbnails for you. You will need a blob trigger and a blob output:

Copy

[FunctionName("CreateThumbnail")]
public static void Run(
    [BlobTrigger("images/{name}")] Stream image,
    [Blob("thumbnails/{name}", FileAccess.Write)] Stream thumbnail,
    ILogger log)
{
  IImageFormat format;

  using (Image<Rgba32> input = Image.Load<Rgba32>(image, out format))
  {
    input.Mutate(x => x.Resize(320, 200));
    input.Save(output, format);
  }  
}

The two image will share the same name, although the container name is different. For the name of the output blob, you could also use some pre-defined functions such as {rand-guid} to generate a unique identifier or {DateTime} to use current time as the name, although this might not match your requirement all the times.

But what if you wanted to write it into the same container? In that case you couldn’t simply use the {name} for your output binding. To generate the name of the output blob at runtime, you need to use imperative bindings.

IBinder

Instead of using the [Blob] attribute, you could use an instance of IBinder interface which gives you much more control over your output binding and it’s name. The same example with a binder will look like:

Copy

using System;
using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Processing;
using SixLabors.ImageSharp.Formats;
using System.Threading.Tasks;
using SixLabors.ImageSharp.PixelFormats;

namespace functions
{
  public static class createThumbnail
  {
    [FunctionName("createThumbnail")]
    public static async Task Run(
        [BlobTrigger("images/original-{blobName}", Connection = "AzureWebJobsStorage")] Stream image,
        string blobName,
        IBinder binder,
        ILogger log)
    {
      IImageFormat format;

      using (Image<Rgba32> input = Image.Load<Rgba32>(image, out format))
      {
        input.Mutate(x => x.Resize(320, 200));

        using (var writer = await binder.BindAsync<Stream>(
                      new BlobAttribute($"images/thumbnail-{blobName}", FileAccess.Write)))
        {          
          input.Save(writer, format);
        }
      }
    }
  }
}

💡 The reason we added a suffix to the blob name is to prevent the infinite look when we write to the same container.

Other bindings

There are heaps of bindings available for you to leverage and create great solutions to solve your business problems and you can find out about them here. All you need to do is to select the Reference > Triggers and bindings from the left hand side.

Summary

Hope this article has been interesting enough to intrigue you to go checkout Azure Functions and leverage their power to solve your business problems. Until next post, Sayōnara 👋🏽.

How to end-to-end test your Vue.js apps with Playwright 🧪

Sun, 07 Feb 2021 00:00:00 GMT

Playwright is one of the recently released end to end testing frameworks which enables fast, reliable and capable automation and is cross platform. I really like it, but since it’s very easy to setup and the community around it is super cool, I like it even more.

In this article I want to show you how you can write some tests for any Vue.js application which is using Auth0 as an identity provider. However, this could be used with any other provider too, since it covers the basics and makes you ready to write tests which cover different scenarios and user interactions.

Concepts

Before we delve into the nitty gritty of things here, we should all agree on a few concepts:

End-to-end tests: End-to-end tests (AKA E2E) are like back box testing where you don’t test individual components or unit of code, instead you focus on testing a scenario end to end. With this type of tests, you use a real instance of the application. They are ideal for creating reliable and bug free application since they mimic user behaviour.
Vue.js: is a fantastic progressive frontend framework which is ideal for building user interfaces. It’s like a middle ground between Angular and React and is built from ground up with developers in mind. It’s easy to pickup and integrate with other libraries or existing projects.
Auth0: is an identity provider which has gained a really good reputation thanks to its complete solution which helps people secure their applications and add features like single sign on, multi-factor authentication and social media login to their applications.

Stage is set

I have an application which is written in Vue.js. I have added authentication and authorization using Auth0 and have different features shown/hidden to users based on their access levels.

However, my unit and component tests don’t seem to cover some scenarios which our end users will do when interacting with our application. Some of this is because I have to use mocks when doing component testing, and unit tests don’t cover more than a piece of code.

Now I need a way to test my application as if a user is sitting in front of their computer and uses our application. To achieve this, I will have to use end-to-end tests.

Options

There are some great E2E test frameworks out there, and here are just a few:

Protractor
Nightwatch.js
Cypress
TestCafe
Playwright
WebdriverJS
OpenTest
Puppeteer

And many more. However, I really like Playwright because it is easy to use and setup, it’s cross platform and integrates nicely with every CI/CD pipeline you’d think of.

The code

So I have an application which basically lists movies and people can buy tickets and go watch it in an imaginary gold cinema. The app also has an admin page where only users with administrator role can access. So let’s break through the code bit by bit:

Main setup

In order for us to use the Auth0 as a plugin with Vue 3 we need to create a plugin and set it up in our main file. However, Vue 3 has changed the way we setup the plugins. So here is our little plugin (note code has been removed for brevity):

Copy

import createAuth0Client from '@auth0/auth0-spa-js';
let client;
///all the other methods and definitions
export const setupAuth = async (options, callbackRedirect) => {
  client = await createAuth0Client({
    ...options,
  });
  try {
    if (window.location.search.includes('code=') 
      && window.location.search.includes('state=')) {
      const { appState } = await client.handleRedirectCallback();
      callbackRedirect(appState);
    }
  }
  //...
  return {
    install: app => {
      app.config.globalProperties.$auth = authPlugin;
    },
  };
}

We also implement a route guard in the same file:

Copy

import { computed, watchEffect } from 'vue';

const authPlugin = {
  isAuthenticated: computed(() => state.isAuthenticated),
  loading: computed(() => state.loading),
  user: computed(() => state.user),
  popupOpen: computed(() => state.popupOpen),
  claims: computed(() => state.claims),
  getIdTokenClaims,
  getTokenSilently,
  getTokenWithPopup,
  handleRedirectCallback,
  loginWithRedirect,
  loginWithPopup,
  logout,
  getUser,
};

export const routeGuard = (to, from, next) => {
  const { isAuthenticated, loading, claims } = authPlugin;
  const verify = () => {
    if (!isAuthenticated.value) {
      return next({ path: '/login', query: { returnUrl: to.path } });
    }

    if (to?.meta?.authorize) {
      const roles = claims.value['http://schemas.microsoft.com/ws/2008/06/identity/claims/role'];
      if (roles.find(r => r === to.meta.authorize.role)) {
        return next();
      } else {
        return next('/unauthorized');
      }
    }
  };
  if (!loading.value) {
    return verify();
  }
  watchEffect(() => {
    if (loading.value === false && claims.value) {
      return verify();
    }
  });
};

This route guard might look intimidating at first glance, but all we’re doing is to create an object which exposes the Auth0 client methods, and then checks the route for a metadata property called authorize which holds the value of the role which should have access to the page.

The rest is just checking whether they match and allow the redirect or send the user to the unauthorized page.

In our main file:

Copy

import { createApp } from 'vue';
import router from './router';
import { setupAuth } from '@/auth/auth-plugin';

const authConfig = {
  domain: process.env.VUE_APP_DOMAIN,
  client_id: process.env.VUE_APP_CLIENTID,
  redirect_uri: process.env.VUE_APP_REDIRECT_URL,
  audience: process.env.VUE_APP_AUDIENCE,
  advancedOptions: {
    defaultScope: 'openid profile email crud:users',
  },
};

function callbackRedirect(appState) {
  router.push(appState && appState.targetUrl ? appState.targetUrl : '/');
}

let app = createApp(App)
  .use(router);

setupAuth(authConfig, callbackRedirect).then(auth => {
  app.use(auth).mount('#app');
});

Here we’re simply creating an options object which is requied by the Auth0 SDK which has the client id, domain etc.

And once that’s done, we will create our app but instead of using the plugin right away, we will call the setupAuth which will then creates the client instance and returns the plugin instance. Now all we need to do is to call the .use and use our plugin instance.

Now that we’ve got our auth plugin setup, it’s time to setup our login component. Fortunately it doesn’t require much code:

Copy

<div v-if="!user">
  <a href="#" class="signup" @click.prevent="login">
    You need to sign in first!
  </a>
</div>

And in our component:

Copy

/// code removed for brevity
export default {
  methods: {
    login: async function() {
      try {
        await this.$auth.loginWithPopup();
        const user = await this.$auth.getUser();
        const accessToken = await this.$auth.getTokenSilently();
        this.$store.commit('SET_USER', user);
        //...
      }
    }
  }
  //...
}

The way this login works is that by clicking on the login button there would be a popup window opened from Auth0 where the user enters their credentials and press submit.

Router config

And the last thing we would have here would be the routing configuration:

Copy

import { createWebHistory, createRouter } from 'vue-router';
import { routeGuard } from '@/auth/auth-plugin';
//other imports
export const routes = [
  {
    path: '/',
    component: Home,
  },
  //...other routes
  {
    path: '/login',
    component: Login,
  },
  {
    path: '/admin',
    component: Admin,
    beforeEnter: routeGuard,
    meta: {
      authorize: {
        role: 'Admin',
      },
    },
  },
  {
    path: '/unauthorized',
    component: UnAuthorized,
  },
];

const router = createRouter({
  history: createWebHistory(),
  routes,
});

export default router;

And that’s the basics of our application. Don’t worry I will put a link to the GitHub repo at the end so you would have all the code. I just want you to know at a really high level how the app is setup.

Setting up the tests

In order to add the package to our app, we will do it via the CLI. So go ahead and execute below command in your terminal at the root of your client app:

Copy

vue add e2e-playwright --dev

It will take a while and a whole bunch of stuff happens behind the scene, but it does all the heavy lifting for you, create a folder for the E2E tests, and even creates a example test for your convenience. It adds Playwright so you can write tests, and chai to handle assertions.

Writing tests

Writing tests is the next step, for each test you have a few basic things to do. Import the necessary objects and methods:

Copy

const { chromium } = require('playwright');
const { expect } = require('chai');

Here I am importing Chrome, but you have the option to use Safari or Firefox if you wish.

Now we need some variables:

Copy

const baseUrl = 'http://localhost:8080/';
const adminPassword = 'Super_Secure_Pass';
const adminUserName = 'admin@example.com';
const normalUserName = 'user@example.com';
const normalUserPassword = 'Super_Secure_Pass';

I am just defining the passwords here to make it easier to understand, you make sure you have them in your environment files and use them that way so that you don’t commit user names and passwords into your source code.

Now it’s time to write our tests, basically you need a describe method which is your test suite. In there you would need two variables for your browser and page instances:

Copy

describe('Authenticated Vue App: ', () => {
  let browser;
  let page;
})

Now you would need to create an instance of your browser and page. So go ahead and add a beforeEach method. Inside that, lunch your browser, create a new page and navigate to your home page:

Copy

before(async () => {
  browser = await chromium.launch();
  page = await browser.newPage();
  await page.goto(baseUrl);
});

Make sure you close these objects at the end of the tests via an after method:

Copy

after(async () => {
  await page.close();
  await browser.close();
});

You’re now ready to write your first test. In this test we’re going to go to admin page without authentication and see what happens. Based on our router guard’s code, we know that the user should be redirected to login:

Copy

it('An unauthenticated user should not be able to see the admin page', async () => {
  await page.goto(`${baseUrl}admin`);
  expect(page.url()).to.equal(`${baseUrl}login?returnUrl=/admin`);
});

If you now run the tests by running yarn test:e2e, you should see the test pass.

More complicated tests

Now to add a spin on our test, say we wanted to actually login and see what happens. In this case we need to click on the login button, then find the opened window and fill in the username and password, then click on submit and come back to our app. This would require a bit more coding, but still easy to find out from Playwright’s documentation.

First you would need to find the login button, then you need to use a Promise.all method to be able to get a reference to your popup window:

Copy

const [popup] = await Promise.all([
  page.waitForEvent('popup'),
  await page.click('a.signup')      
]);

Now that you have the reference, you need to fill in the info and click on the login:

Copy

await popup.fill('input[type="email"]', adminUserName);
await popup.fill('input[type="password"]', adminPassword);
await popup.click('button[type="submit"]');

And at last you need to make an assertion. Say you wanted to see whether an admin user will have access to the admin page. To do the assertion, you need to hook up to the close event of the popup window. So your test will look like:

Copy

it('be redirected back to admin page after login', async () => {
    await page.goto(`${baseUrl}admin`);
    
    const [popup] = await Promise.all([
      page.waitForEvent('popup'),
      await page.click('a.signup')      
    ]);

    popup.on('close', async () => {
      expect(page.url()).to.equal(`${baseUrl}admin`);
    });

    await popup.fill('input[type="email"]', adminUserName);
    await popup.fill('input[type="password"]', adminPassword);
    await popup.click('button[type="submit"]');
  });

The reason why you’d need a waitForEvent method in the Promise.all method is that you need to wait for the popup window to be able to get a handle on it. Now if you run the tests again, they should all pass.

Full code

You can find the full source code on my GitHub repository here.

Summary

And that’s how easy it is to write tests which mimic user interactions and can make you confident to ship reliable software. Happy testing and let me know what could automation have you done with Playwright if you got to that point 👋🏽👋🏽.

How much do you know about Azure CosmosDB Emulator? 🧐

Tue, 15 Dec 2020 00:00:00 GMT

Azure Cosmos DB is one of the foundational services in Azure, which provides high availability, global scale and an impressive performance. However, it could be a bit costly for developers to spin up an instance during their development especially if they don’t know some of the basics such as what should be the partition key, what throughput they should select and so on that much.

The Azure Cosmos DB Emulator is a service provided by Microsoft which allows you to emulate the Cosmos DB Service locally for development purposes. In addition, no matter whether you’re using Azure CosmosDB or not, at some point you might have to test some sort of DocumentDB locally.

Intro

Let’s be honest, there are other ways to work with Cosmos DB without paying any money such as a Free Trial Account which gives you $200 dollars to spend over a year, or even the Cosmos DB Query Playground which is an interactive website which helps you to learn the query syntax and provides you with a set of pre-defined JSON documents to work with.

But eventually you either run out of money, or your product is live and your developers want to develop new features and fix bugs to be released very quickly. That’s when Cosmos DB Emulator really comes to the rescue. You have other advantages such as ability to work offline and or even have an Azure subscription.

Features

API set

Azure Cosmos DB supports SQL API, as well as Table, MongoDB, Cassandra, and Gremlin which to me covers many scenarios from migrating an existing database from on-premise to Azure, modernising your legacy application, or create a brand new one. Azure Cosmos DB Emulator also supports these APIs although the underlying implementation details are different to the real service on Azure. In fact the emulator hides all abstractions from you and lets you focus on adding value to your team or organisation.

💡 However keep in mind that the data explorer provided with the service only supports SQL API at the moment.

Number of containers

The emulator allows you to create a single account with multiple (up to 25 fixed sized, or 5 unlimited) containers.

Operating System

At the moment, the emulator only supports 64-bit versions of Windows (Server 2012 R2, Server 2016, Server 2019, Windows 10) which demands a minimum of 2GB of RAM and 10GB of free disk space. You must also have administrative privileges to be able to install the client.

In addition to that, it also comes with docker image which you can use to run the emulator in a docker container but only on windows containers. If you want to use docker in Linux or macOS, you will need to use it on a Windows virtual machine.

Installation

Simply download the executable from here and install it on your Windows OS, if you’re running Linux or macOS or want to run it in a docker container please refer to the official documentation.

Differences

As amazing it looks, there are a few differences you need to keep in mind when working with the emulator:

As I mentioned before the data explorer only supports the SQL API. But you can use all other APIs from within your application using the SDK.
It only supports a single account with a well known key. You can change the key from command line, but you cannot change it in the explorer.
For now it only supports provisioning throughput mode only, meaning you can’t use it in serverless mode.
It does not support core features such as different consistency levels, scaling, replication and performance guaranties.
Since the client might not be up to date with Azure Cosmos DB service, make sure you always check the Azure Cosmos DB capacity planner to accurately estimate your required throughput.
And the minimum ID property size is 256 characters.

Summary

In this short article I just showed you the tip of the iceberg about what the Azure Cosmos DB Emulator has to offer for you. It’s a great utility to be able to develop applications who rely on Azure Cosmos DB locally and has helped me in my pet projects which doesn’t even need to work with a Cosmos DB instance. Keep an eye out for more articles on some use cases this amazing tool can help you with.

Add dual screen support to your applications 💻

Mon, 07 Dec 2020 00:00:00 GMT

If you have ever tried to add responsiveness to your web application you would know it’s difficult or at least takes a while to get it right. As progressive web apps (PWAs) are the first class citizens on many devices with the ability to get installed, it’s even more important to make sure user’s have the best experience possible. But technology advances in a much faster pace. We have now foldable devices like the shiny new Microsoft Surface Duo which add more complexity to this equation.

Problem

What if you wanted to show your page across two screens without hiding the content under the middle edge? At least for now we don’t have any device with bending glass to be able to not worry about what appears where on the screen.

In order to continue with this we need a demo application to make sure we’re on the same page. So let’s create one, say we have an image gallery which has 5 columns and someone is looking at it on their brand new Surface Duo:

Copy

<div class="gallery">
  <div class="grid">
    <div class="card">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=5"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
    <div class="card p-3">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=4"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
    <div class="card">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=3"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
    <div class="card">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=2"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
    <div class="card">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=1"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
    <div class="card">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=7"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
    <div class="card">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=8"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
    <div class="card">
      <img
        class="card-img-top"
        src="https://picsum.photos/200/200?random=9"
        alt="Card image cap"
      />
      <div class="card-body">
        <h5 class="card-title">Card title</h5>
        <p class="card-text">
          This card has supporting text below as a natural lead-in to
          additional content.
        </p>
        <p class="card-text">
          <small class="text-muted">Last updated 3 mins ago</small>
        </p>
      </div>
    </div>
  </div>
</div>

And in our CSS file we just put a 5 column layout for the card column like so:

Copy

.grid {
  display: grid;
  grid-template-columns: repeat(5, 1fr);
  grid-row: auto;
}

Don’t worry too much about the code, you can view the result here on Glitch. Now to test how this page looks like you either have to have the device handy, or if you don’t have it like me, just use the Surface Duo Emulator which I find really cool. For example, here is how the page looks like at the moment:

You can see right away that it’s not a good look, I mean who would be willing to spend a second on this site? So how do we fix it? I think we’re now on the same page, so let’s dive in.

Solution

But don’t you worry, we’ve got you covered by two brand new experimental APIs that will enable you to effectively lay your elements on the screen where you want them to be.

The former along with a set of environment variables allows you to be aware of the geometry of the fold. The latter is an API which can be used alongside the media feature to work with non-DOM targets like Canvas 2D and WebGL.

Detecting display regions

As I mentioned earlier, the **screen-spanning** media feature allows you to be aware of the fact that the page is spanned across multiple display regions. But how do we use it? Let’s have a look at the syntax. At its core, you will have the ability to use the below media query:

Copy

@media (screen-spanning: <value>) { }

The allowed values are:

single-fold-vertical: which describes the state of the viewport, when it’s spanning across a single fold and the fold’s posture is vertical. This would mean the device is in double portrait mode like below:

Image from Microsoft Surface Duo blog

The code for this mode would be:

Copy

@media (screen-spanning: single-fold-vertical) { 
  /* styles applied in double-portrait (wide) mode */ 
}

single-fold-horizontal: which describes the state of the viewport, when it’s spanning across a single fold but this time the posture is horizontal. This would match the double portrait tall mode like below:

Image from Microsoft Surface Duo blog

Copy

@media (screen-spanning: single-fold-horizontal) {
   /* styles applied in double-landscape (tall) mode */
}

Environment variables

To get the geometry of the device fold, you could leverage the browser environment variables. You have access to four environment variables:

fold-top
fold-left
fold-width
fold-height

The picture below would nicely show you what there values represent:

Image from Microsoft Surface Duo blog

So let’s see how we can use these features to improve our user experience a bit. First we know that we’re using the single-fold-vertical mode, so we’ll add the relevant media query and play with the column count trying to either put the columns in the left or the right side of the fold:

Copy

@media (screen-spanning: single-fold-vertical) { 
.card-columns { 
  grid-template-columns: repeat(4, calc( env(fold-left) /2 - 10px) ); 
  } 
}

💡 In order to access these features, you must enable experimental platform features on Edge by visiting edge://flags/#enable-experimental-web-platform-features and enabling them.

After setting the column count to 4, we will end up with something like this:

So we’re one step close to have a better view, but we have the two middle columns right on the fold, and the title is spanning across too. Let’s fix the title first. We could simply set a width for it to be in the first fold, or we could give it a margin right calculating where the fold is located. Let’s use the margin solution.

Copy

h1 { 
  margin-right: calc( 100vw - (env(fold-left) + env(fold-width)) ); 
}

Here we’re calculating the width of the second fold and setting that as margin right for our header. Of course there are many other ways to achieve this, but I wanted to demonstrate the use of environment variables. After applying the above styles, we will end up with below, which is one step ahead to have a better UX.

Now is the time to fix the two middle columns. One way to fix the middle column problem is to add an empty column which has no content. But it’s easier said than done. If you’re willing to play with the width of the columns, you can get away with something like this:

Copy

@media (screen-spanning: single-fold-vertical) {
  .grid {    
    grid-template-columns: repeat(4, 23%);
    grid-gap: 25px;
  }
  .card:nth-child(4n + 2) {
    margin-right: 5px;
  }
  .card:nth-child(4n + 3) {
    margin-left: 10px;
  }
}

This will result in:

You can see we’ve come a long way from having a completely spanned content to a neat ordered view. But that’s all have been using only the CSS media feature. Let’s see what does the Window Segment Enumeration API have to offer.

Window Segments Enumeration API

This API helps you to deal with non-DOM elements such as a canvas or WebGL objects. You can leverage the getWindowSegments() function which is accessible on the window object and returns an array of one or more DOMRect representing the geometry of and position of each display region:

Copy

const segments = window.getWindowSegments();

// case 1: desktops, traditional touch screen devices, foldable device not spanned
console.log(segments.length) // 1

// case 2: dual-screen and foldable 
console.log(segments.length) // 2

If you wanted to know the fold posture inside your JavaScript code, you can calculate them easily using below snippet:

Copy

function isSingleFoldHorizontal() {
    const segments = window.getWindowSegments();
    if( segments.length !== 2 ) {
        return false;
    }
    if( segments[0].top < segments[1].top ) {
        return true;
    }
    return false;
}

Here we’re simply retuning false if the segments are now two, then check whether the top value of segment 0 is less than segment 1 which means the device is in horizontal state. If not we’re returning false which means it’s vertical posture.

Where to start?

You can access the CSS screen-spanning media features using the experimental web platform feature flag, and the JavaScript Window Segments Enumeration API is accessible by enabling the Experimental JavaScript flag.

You also have the option of using Origin Trials, where you can acquire tokens and safely experiment with these new primitives in production in exchange for providing feedback about the APIs. Sign up for an Origin Trial if you’re interested in testing out these APIs!

PS: You can find all the the code on Glitch here.

How to use Azure Cosmos Emulator as a local MongoDb database 🌍

Mon, 30 Nov 2020 00:00:00 GMT

I recently was trying to prepare a demo which involved me having a local MongoDb database. I reviewed a few options and was about to choose one when I remembered I have Azure Cosmos DB Emulator installed and Cosmos DB supports MongoDb APIs.

Install if you haven’t already

You will need below requirements to be able to install the emulator:

Windows Server 2012 R2, Windows Server 2016, 2019 or Windows 8 and 10. Docker on Windows, Linux and macOS is also supported.
64-bit OS.
Minimum 2GB RAM.
At least 10GB available hard disk space.

If you’ve ticked all of above already, then head over here and install the latest version. If you stumbled upon any issues use the troubleshooting guide to find out what’s happening.

💡 Make sure you check for updates regularly since each new version might contain new features and bug fixes.

Starting the emulator

Normally you would use the Windows start menu to find your programs and start the application, however, for the emulator to support MongoDB APIs, you will need to pass the EnableMongoDbEndpoint argument.

You have a few options here:

Create a shortcut and pass the required arguments
Use command line to start it

I am going to use Windows Terminal. Run the below command to fire up the emulator with MongoDB support:

Copy

& 'C:\Program Files\Azure Cosmos DB Emulator\CosmosDB.Emulator.exe' /EnableMongoDbEndpoint

You will be prompted to approve the emulator making changes to your system and once approved, you will see the emulator starting and the interface will open in your default browser.

Getting the connection string

The very first thing you’d see is the quick start which should look something like this:

What you’ll need is the Mongo Connection String. Copy that you’ll be ready to kick start. I am going to use the Node sample from the quick start to continue the rest of this guide, but you can use this straight away in your current code.

Creating a Node.js Web API project

Open up a command prompt in a new folder and initiate a project:

Copy

npm init -y

The next thing to do is to install the dependencies:

Copy

# linux or macOS
touch app.js

# Windows
echo "" > app.js

# install dependencies
npm install express mongodb body-parser

The app

We would have a simple application, since this tutorial is not a Node.js or Express.js focused, I will just brush on some steps here. Feel free to go through and understand each of these at your own pace.

Copy

const Express = require("express");
const BodyParser = require("body-parser");
const MongoClient = require("mongodb").MongoClient;
const ObjectId = require("mongodb").ObjectID;
var app = Express();
app.use(BodyParser.json());
app.use(BodyParser.urlencoded({ extended: true }));
app.listen(5000, () => {});

What we have here is a simple web server which responds to http://localhost:5000. All we’re doing is initializing the MongoClient and a web server. Next thing we need to do is to establish a connection with MongoDB interface of our emulator:

Copy

const Express = require("express");
const BodyParser = require("body-parser");
const MongoClient = require("mongodb").MongoClient;
const ObjectId = require("mongodb").ObjectID;
const CONNECTION_URL = "YOUR CONNECTION STRING";
const DATABASE_NAME = "cosmos_emulator_mongo";


var app = Express();
app.use(BodyParser.json());
app.use(BodyParser.urlencoded({ extended: true }));
var database, collection;

app.listen(5000, () => {
    MongoClient.connect(CONNECTION_URL, { useNewUrlParser: true }, (error, client) => {
        if(error) {
            throw error;
        }
        database = client.db(DATABASE_NAME);
        collection = database.collection("celebrities");
        console.log("Connected to `" + DATABASE_NAME + "`!");
    });
});

The next thing we do to make it simpler to run the app is to add a start script to our package.json:

Copy

"scripts": {
    "start": "node app.js"
}

If you run npm start now, you should see the message connected to Connected to 'cosmos_emulator_mongo' printed on your console which means you’ve successfully connected to the MongoDb APIs.

Seed the database

First let’s create a JSON file which contains some celebrities 😎:

Copy

[{
  "age": 18,
  "birth_place": "Harij, Gujarat, India",
  "birth_sign": "Pisces",
  "birth_year": "1996",
  "birthday": "September 28",
  "name": "Daxeel Soni",
  "occupation": "Developer of this API. More: www.daxeelsoni.in",
  "photo_url": "http://daxeelsoni.in/images/me.jpg"
}, {
  "age": 28,
  "birth_place": "Houston",
  "birth_sign": "Libra",
  "birth_year": "1987",
  "birthday": "September 28",
  "name": "Hilary Duff",
  "occupation": "TV Actress",
  "photo_url": "http://www.famousbirthdays.com/thumbnails/duff-hilary-medium.jpg"
}, {
  "age": 38,
  "birth_place": "Columbia",
  "birth_sign": "Libra",
  "birth_year": "1977",
  "birthday": "September 28",
  "name": "Young Jeezy",
  "occupation": "Rapper",
  "photo_url": "http://www.famousbirthdays.com/thumbnails/jeezy-young-medium.jpg"
}, {
  "age": 48,
  "birth_place": "Roanoke",
  "birth_sign": "Libra",
  "birth_year": "1967",
  "birthday": "September 28",
  "name": "Challen Cates",
  "occupation": "TV Actress",
  "photo_url": "http://www.famousbirthdays.com/thumbnails/cates-challen-medium.jpg"
}]

Now that we have our JSON file, let’s seed the database. Replace the initial connect code with this:

Copy

app.listen(5000, () => {
  MongoClient.connect(
    CONNECTION_URL,
    {
      useNewUrlParser: true,
      sslValidate: false,
    },
    (error, client) => {
      if (error) {
        throw error;
      }
      database = client.db(DATABASE_NAME);
      collection = database.collection("celebrities");
      collection.estimatedDocumentCount({}, (erorr, numOfRecords) => {
        if (numOfRecords <= 0) {
          fs.readFile("./info.json", "utf8", (err, data) => {
            if (err) {
              console.log(`Error reading file from disk: ${err}`);
            } else {
              // parse JSON string to JSON object
              const celebrities = JSON.parse(data);

              collection.insertMany(celebrities, (error, result) => {
                if (error) {
                  console.log(`Error in saving seed data: ${error}`);
                }

                console.log(`Seed data inserted into the database!`);
              });
            }
          });
        } else {
          console.log(`Connected to database`);
        }
      });
    }
  );
});

All we’re doing here is to get the number of records in the collection and if there is nothing, read the JSON file and write it to the collection.

Don’t forget to add const fs = require('fs') at the top with other require statements.

Now if you run npm start would should see the application start and then the seed data inserted message.

Creating the endpoint

And now all we need is to add our get endpoint to be able to fetch our celebrities. Don’t forget to add it before you create your listener:

Copy

app.get("/api/celebrities", (request, response) => {
    collection.find({}).toArray((error, result) => {
        if(error) {
            return response.status(500).send(error);
        }
        response.send(result);
    });
});

Testing the application

You’re now ready to get your celebrities via the API. Simply open a browser and head over to http://localhost:5000/api/celebrities or just use the below command:

Copy

curl -X GET http://localhost:5000/api/celebrities

Summary

In this guide we saw how to use Azure Cosmos DB Emulator with MongoDb API to have a local MongoDb available for our local development and proof of concepts. Hope this has helped you save some time and also the trouble of installing yet another software on your system (of course if you already are using the emulator 😊).

Enjoy hacking and let us know what awesome projects you’re doing using the emulator.

PS: You can find the finished code in my GitHub repository here.

Access your clipboard on your other devices 😍 📋

Fri, 13 Nov 2020 00:00:00 GMT

How many times have you been working on multiple systems and realised you are trying to paste something which you have copied on the other system? Well with the help of this article you can now share your clipboard between your Windows and Mac devices 😎.

Windows to Windows

If you have multiple windows devices, it’s very easy. Simply open your control panel and search for Clipboard Settings.

Once opened, set the Sync across devices option to on. If you want to prevent sensitive data to be synced and do it on a case by case basis, set the Never automatically sync text that I copy option to on instead.

While you’re at it, turn clipboard history on too to have a beautiful clipboard history available by pressing Win+V.

Use the copied text

Now if you copy something into your clipboard, based on assuming you wanted to sync everything, simply paste, or open your clipboard history on the target device to see it.

⚠️ If you selected never automatically sync option, you can press Win+V to open up your clipboard history and select what you want to be synced across your devices after copying the items.

Additional tip

If you choose to use clipboard history, make sure you pin items you use frequently to save yourself some more time too.

Mac to Mac

If you’re a Mac user and want to achieve the same, you can use the Universal Clipboard.

It’s really simple, like we saw on windows setup.

Windows/Mac/Linux to Mac/Windows/Linux

If you use two different operating systems and want to simply share your content across, nothing works better than PASTEBIN.

This is a web tool that allows you to share your content across different devices with ease.

Warning

When using third party software whether it’s application or web app, never share secrets or your personal information.

Summary

We saw how to take some simple steps to increase our productivity and access any content we would like across our devices. Hope you benefit from this like I did and see you next time.

I've joined Microsoft as an Azure Technical Trainer 🗺️ 😍

Mon, 02 Nov 2020 00:00:00 GMT

The cat is out of the bag now, I’ve joined Microsoft as an Azure Technical Trainer. Some of you might be wondering why, and if so, this post is for you. I’ve decided to write down why I’ve made this decision and a bit about my 15+ years journey from high school to where I am now.

PS: We don’t have a Microsoft campus in Melbourne, but it didn’t prevent me photoshopping myself in 😜. Also, Satya Nadella was another motivation for me to join since we have a decent resemblance 😎.

Background

I started to write small programs when my brother bought a PC back in 1998. It was running MS-DOS and the first program I wrote could print a pyramid with stars on the console and was written in QBasic. From then, I was always amazed by people who write programs which can help others write their own, and working at Microsoft became one of my dream jobs.

Later on, I progressed through high school, pre-uni and then uni when I started my journey as a software engineer. I’ve started with C++, Turbo Pascal, FoxPro, Delphi, ASP Classic, PHP, and many other languages and ended up with, and finally ASP.Net MVC.

Throughout those years I became really interested to web development and have been active in that area so far. Of course that wasn’t the only thing I picked up, I have written windows desktop apps with ASP.Net web forms and Java Spring, Java Card applets to work with HSM and physical security devices, ActiveX, and many more I can’t even remember, not because I didn’t like them, but because I was really into learning new things and there were too many out there.

I wasn’t born in a wealthy family, so I had work summers and other free times to earn enough to go through my studies for the rest of the year. And we don’t have big companies represented in Iran, so working for Microsoft was became a wish in my bucket list.

Post uni

When I graduated from University with a Master’s degree in Information Security, I was hired by one of the well known software companies who were involved in banking industry. From there, I had jobs as software engineer, team lead and line manager and eventually I became the software engineering lead for one of the private banks’s PSP (payment service provider) company. But throughout all those years, I was on the lookout for an opportunity to migrate to another country and continue my professional career there.

Migrating to Australia

We considered Canada, US, and a few countries in Europe, but ended up pinning Australia on the map because of a few different reasons which suited our lifestyle and goals. So we sold/gave away most of our household items, rented out the apartment and packed our suitcases, jumped on a plane and ended up in Melbourne, Australia in 2015. I don’t want to go through how we got the Skilled Visa because it’s simply too long and brings back many not so good memories.

First job here

I immediately started applying for jobs here and without surprise, I was getting many calls, but not an actual opportunity that I liked. After one week, I had to bring down my expectations and start from scratch. I applied for a mid-level developer at a company and got in. This was after 3 weeks from our arrival and believe it or not is a record time for migrants to get a job.

Moving into consulting and public speaking

I didn’t stay there for long, as I didn’t want to waste any time following my goals. So I got into a consulting firm and eventually joined Readify (one of the best consulting firms in Australia) back in 2016. Because of my love for knowledge sharing, I got into public speaking starting from local meetups ended up at international conferences around the world. I am super proud of what I’ve achieved throughout these years and owe a big part to Readify and later on Telstra Purple. However, I still had my bucket list getting dust and wasn’t enjoying what I was doing as my full time job.

Decision to shift to developer advocacy

I was doing conference talks, blogs, and was involved in development community which has been a real pleasure, but managing all of that while having a full time job was really tough. So I started to search for similar roles and came across developer advocacy. However, this not a role which is very well known in Australia apart from big companies like Microsoft, Amazon AWS, IBM etc and some smaller ones like Auth0 and Twilio. I had a few discussions with different people to get an idea of what’s required to be able to score one of those jobs, but didn’t get a chance to find and apply for one.

When an opportunity presents itself

So I continued what I was doing, but still were on the lookout for something interesting which could help me do what I love to do. A few weeks back one of my friends who also works at Microsoft told me about a position on the Azure Technical Training team and I thought that’s something I would be really good at. It would be a bit of shift from web development, but the pros (working at Microsoft, sharing knowledge, and being close to what I love to do) convinced me to apply. I went through the interview and got an offer which made my year. This year has been really tough for me and my family and this event really helped us recover from these last few months of stress and anxiety.

Summary

So here I am, employed by my dream company, and doing what I love to do. It couldn’t have ended up better than this, I’ve already talked to my manager and he knows and supports my passion to be involved in development community, so this doesn’t mean I won’t be active on conferences and blog posts anymore. This amazing opportunity will help me grow in many ways as working at Microsoft and working with so many talented and amazing people would.

I just hope this has inspired a few of you fellow developers out there to never give up hope and follow your dreams however distant and impossible they seem 🤘🏽 💪🏽.

Get started with Vue 3 and Tailwindcss

Thu, 22 Oct 2020 00:00:00 GMT

Last week I wanted to setup a Vue.js v3 app with Tailwindcss and although many articles exists for that, I couldn’t get it done. The fact is that you need to use postcss to get your setup working, but with new version of Vue, the postcss config file is not picked up. So after a few try and errors, I finally got it working and thought to jot down what I went through to make it easier for my future self, and hopefully a few of my fellow developers around the world.

Vue CLI

You can setup your Vue project with simply importing the script tag and start coding, but I normally like to use Vue CLI to get started because it takes care of the many things for me and gives a really good starting point.

So let’s start by installing Vue CLI if you don’t have it already:

Copy

npm install -g @vue/cli
# OR
yarn global add @vue/cli

This will install the Vue CLI for you and once that’s done, you’d be ready to create your project. If you already have CLI installed, make sure you update it first to get support for Vue v3 preview.

Creating the project

In order to create your project, you need to call the CLI and give your project name:

Copy

vue create vue-tailwindcss

This command will start the wizard and asks you what version of Vue you want to use and what additional options you want to have.

You can choose between default seetings, or making a completely custom setup. I will normally choose custom which gives me more freedom as to what I want to setup.

From here press Entre on Choose Vue version, and select 3.x (Preview).

I would normally choose TypeScript, Babel, Linter, Unit and E2E testing optiuons, but feel free to setup how you like.

When it’s done go navigate into the folder or open it up with your editor of choice. Mince is VS Code.

Installing the required dependencies

At this point we need to install Tailwindcss:

Copy

npm install tailwindcss

Create your style file

At this stage you’re ready to create your style file. I will put it next to my main.js for convinience.

Don’t forget to import it inside your main.js file:

Copy

import "./main.css";
// ...

And now is the time to import the Tailwincss base and components in our css file:

Copy

/* src/main.css */
@import "tailwindcss/base";
@import "tailwindcss/components";
@import "tailwindcss/utilities";

Postcss configuration

Many articles tell you to create a postcss.config.js or .postcssrc.js and set your config there, but with the new version of Vue CLI this doesn’t get picked up. For this part we simply need to update our package.json file:

Copy

"postcss": {
  "plugins": {
    "tailwindcss": {},
    "autoprefixer": {}
  }
}

Import Tailwindcss components

You’re ready to use the Tailwindcss components now. So open up your hello-world.vue file and paste this code from their free gallery in:

Copy

<!--
  Tailwind UI components require Tailwind CSS v1.8 and the @tailwindcss/ui plugin.
  Read the documentation to get started: https://tailwindui.com/documentation
-->
<div class="relative bg-white overflow-hidden">
  <div class="max-w-screen-xl mx-auto">
    <div class="relative z-10 pb-8 bg-white sm:pb-16 md:pb-20 lg:max-w-2xl lg:w-full lg:pb-28 xl:pb-32">
      <svg class="hidden lg:block absolute right-0 inset-y-0 h-full w-48 text-white transform translate-x-1/2" fill="currentColor" viewBox="0 0 100 100" preserveAspectRatio="none">
        <polygon points="50,0 100,0 50,100 0,100" />
      </svg>

      <div class="relative pt-6 px-4 sm:px-6 lg:px-8">
        <nav class="relative flex items-center justify-between sm:h-10 lg:justify-start">
          <div class="flex items-center flex-grow flex-shrink-0 lg:flex-grow-0">
            <div class="flex items-center justify-between w-full md:w-auto">
              <a href="#" aria-label="Home">
                <img class="h-8 w-auto sm:h-10" src="https://tailwindui.com/img/logos/workflow-mark-on-white.svg" alt="Logo">
              </a>
              <div class="-mr-2 flex items-center md:hidden">
                <button type="button" class="inline-flex items-center justify-center p-2 rounded-md text-gray-400 hover:text-gray-500 hover:bg-gray-100 focus:outline-none focus:bg-gray-100 focus:text-gray-500 transition duration-150 ease-in-out" id="main-menu" aria-label="Main menu" aria-haspopup="true">
                  <svg class="h-6 w-6" stroke="currentColor" fill="none" viewBox="0 0 24 24">
                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h16" />
                  </svg>
                </button>
              </div>
            </div>
          </div>
          <div class="hidden md:block md:ml-10 md:pr-4">
            <a href="#" class="font-medium text-gray-500 hover:text-gray-900 transition duration-150 ease-in-out">Product</a>
            <a href="#" class="ml-8 font-medium text-gray-500 hover:text-gray-900 transition duration-150 ease-in-out">Features</a>
            <a href="#" class="ml-8 font-medium text-gray-500 hover:text-gray-900 transition duration-150 ease-in-out">Marketplace</a>
            <a href="#" class="ml-8 font-medium text-gray-500 hover:text-gray-900 transition duration-150 ease-in-out">Company</a>
            <a href="#" class="ml-8 font-medium text-indigo-600 hover:text-indigo-900 transition duration-150 ease-in-out">Log in</a>
          </div>
        </nav>
      </div>

      <!--
        Mobile menu, show/hide based on menu open state.

        Entering: "duration-150 ease-out"
          From: "opacity-0 scale-95"
          To: "opacity-100 scale-100"
        Leaving: "duration-100 ease-in"
          From: "opacity-100 scale-100"
          To: "opacity-0 scale-95"
      -->
      <div class="absolute top-0 inset-x-0 p-2 transition transform origin-top-right md:hidden">
        <div class="rounded-lg shadow-md">
          <div class="rounded-lg bg-white shadow-xs overflow-hidden" role="menu" aria-orientation="vertical" aria-labelledby="main-menu">
            <div class="px-5 pt-4 flex items-center justify-between">
              <div>
                <img class="h-8 w-auto" src="https://tailwindui.com/img/logos/workflow-mark-on-white.svg" alt="">
              </div>
              <div class="-mr-2">
                <button type="button" class="inline-flex items-center justify-center p-2 rounded-md text-gray-400 hover:text-gray-500 hover:bg-gray-100 focus:outline-none focus:bg-gray-100 focus:text-gray-500 transition duration-150 ease-in-out" aria-label="Close menu">
                  <svg class="h-6 w-6" stroke="currentColor" fill="none" viewBox="0 0 24 24">
                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12" />
                  </svg>
                </button>
              </div>
            </div>
            <div class="px-2 pt-2 pb-3">
              <a href="#" class="block px-3 py-2 rounded-md text-base font-medium text-gray-700 hover:text-gray-900 hover:bg-gray-50 focus:outline-none focus:text-gray-900 focus:bg-gray-50 transition duration-150 ease-in-out" role="menuitem">Product</a>
              <a href="#" class="mt-1 block px-3 py-2 rounded-md text-base font-medium text-gray-700 hover:text-gray-900 hover:bg-gray-50 focus:outline-none focus:text-gray-900 focus:bg-gray-50 transition duration-150 ease-in-out" role="menuitem">Features</a>
              <a href="#" class="mt-1 block px-3 py-2 rounded-md text-base font-medium text-gray-700 hover:text-gray-900 hover:bg-gray-50 focus:outline-none focus:text-gray-900 focus:bg-gray-50 transition duration-150 ease-in-out" role="menuitem">Marketplace</a>
              <a href="#" class="mt-1 block px-3 py-2 rounded-md text-base font-medium text-gray-700 hover:text-gray-900 hover:bg-gray-50 focus:outline-none focus:text-gray-900 focus:bg-gray-50 transition duration-150 ease-in-out" role="menuitem">Company</a>
            </div>
            <div>
              <a href="#" class="block w-full px-5 py-3 text-center font-medium text-indigo-600 bg-gray-50 hover:bg-gray-100 hover:text-indigo-700 focus:outline-none focus:bg-gray-100 focus:text-indigo-700 transition duration-150 ease-in-out" role="menuitem">
                Log in
              </a>
            </div>
          </div>
        </div>
      </div>

      <main class="mt-10 mx-auto max-w-screen-xl px-4 sm:mt-12 sm:px-6 md:mt-16 lg:mt-20 lg:px-8 xl:mt-28">
        <div class="sm:text-center lg:text-left">
          <h2 class="text-4xl tracking-tight leading-10 font-extrabold text-gray-900 sm:text-5xl sm:leading-none md:text-6xl">
            Data to enrich your
            <br class="xl:hidden">
            <span class="text-indigo-600">online business</span>
          </h2>
          <p class="mt-3 text-base text-gray-500 sm:mt-5 sm:text-lg sm:max-w-xl sm:mx-auto md:mt-5 md:text-xl lg:mx-0">
            Anim aute id magna aliqua ad ad non deserunt sunt. Qui irure qui lorem cupidatat commodo. Elit sunt amet fugiat veniam occaecat fugiat aliqua.
          </p>
          <div class="mt-5 sm:mt-8 sm:flex sm:justify-center lg:justify-start">
            <div class="rounded-md shadow">
              <a href="#" class="w-full flex items-center justify-center px-8 py-3 border border-transparent text-base leading-6 font-medium rounded-md text-white bg-indigo-600 hover:bg-indigo-500 focus:outline-none focus:border-indigo-700 focus:shadow-outline-indigo transition duration-150 ease-in-out md:py-4 md:text-lg md:px-10">
                Get started
              </a>
            </div>
            <div class="mt-3 sm:mt-0 sm:ml-3">
              <a href="#" class="w-full flex items-center justify-center px-8 py-3 border border-transparent text-base leading-6 font-medium rounded-md text-indigo-700 bg-indigo-100 hover:text-indigo-600 hover:bg-indigo-50 focus:outline-none focus:shadow-outline-indigo focus:border-indigo-300 transition duration-150 ease-in-out md:py-4 md:text-lg md:px-10">
                Live demo
              </a>
            </div>
          </div>
        </div>
      </main>
    </div>
  </div>
  <div class="lg:absolute lg:inset-y-0 lg:right-0 lg:w-1/2">
    <img class="h-56 w-full object-cover sm:h-72 md:h-96 lg:w-full lg:h-full" src="https://images.unsplash.com/photo-1551434678-e076c223a692?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=2850&q=80" alt="">
  </div>
</div>

Run the application

You’re all set, run npm run serve and when the CLI is finished compiling the app, open a browser window and head to https://localhost:8080 to see the Tailwindcss component:

Customisation

If you want to customise any of the default styles, you need to modify the theme. To do so, run:

Copy

npx tailwind init

This will create a tailwind.config.js file for you in the root of your directory. To demo an example customisation, let’s say we want to add some space between the photo and the right hand side of the page. Currently the photo is having an position:absolute and right:0 because it has the lg:right-0 class.

So we want to add a new class which has 10% margin on the right hand side of the image. For that to work, we will need below code in our tailwindcss.config.js:

Copy

module.exports = {
  future: {
    // removeDeprecatedGapUtilities: true,
    // purgeLayersByDefault: true,
  },
  purge: [],
  theme: {
    inset: {
      '0': 0,
      auto: 'auto',
      '1/10': '10%',
    },
    extend: {},
  },
  variants: {},
  plugins: [],
}

Pay attention that I’ve added a 1/10 with the value of 10% which we can use. Now all we need to do is to add lg:right-1/10 to the image container which pushes the image to the left by 10%:

Copy

<div class="lg:absolute lg:inset-y-0 lg:right-1/10 lg:w-1/2">
</div>

And now it should look like this:

Summary

We saw how to add Tailwindcss library to our Vue 3 application created by Vue CLI. Hopefully this will help some of you out there facing the same issue as me. See you next time 👏🏽.

You can find the code in my GitHub repository.

Do you know about PowerToy Utilities? 💪🏻

Fri, 09 Oct 2020 00:00:00 GMT

If you’re a windows user like me, you might have heard of PowerToys Utilities from Microsoft. They are a set of utilities which come handy in many situations, but not many people know and use them. So I thought let’s write how and when I use them to help other people be more efficient too.

PowerToys utilities

Microsoft PowerToys are a set of small applications which helps windows users streamline their day to day activities and windows experience for greater productivity. However, these are not new. There were a set of tools in Widows 95 (yes I’m that old) which inspired this project.

Installation

Simply head to the Downloading and Release notes section on their GitHub repository, download the version you’re after and install it using its installer.

The latest version as of writing this article is v0.23.0. Once you get it installed, it will update itself on a regular basis.

So let’s see what are these and how you can use them.

Color picker

Color Picker is a handy little tool which allows you to pick a color from anywhere on your screen. As a web developer there are many times when I want to get the hex or rgb value of a background color, border, or other areas.

I used to use browser DevTools, but they only help you within the browser and if you want something from a document, picture or other files on your file system, then this tool will come in handy.

If you haven’t customised the default shortcuts, press Win + Shift + C to open the picker. Once opened, you can use your mouse scroll to zoom in and left click to copy the value to your clipboard.

Fancy zones

Fancy Zones allows you to create complex window layouts and quickly position your windows into those. This comes really handy if you have a super wide monitor setup.

To open it, press Win + ` and select your desired layout from the opened window. Once selected you can arrange your windows into those areas by pressing Win + ⬅ or Win + ➡.

💡 If the arrow keys didn’t work for you, simply select the option to override windows shortcuts in the settings. You can access the settings from clicking the PowerToys icon in your toolbar.

File explorer add-ons

File Explorer add-ons will enable SVG icon rendering and markdown preview for your file explorer. To see your preview pane simply click the view menu and click the preview pane.

Image resizer

Image Resizer is a really cool shell extension which allows you to quickly resize images. Simply right click on your image or images and resize them instantly.

keyboard manager

Keyboard manager allows you to create your own keyboard layout to increase your productivity. You can map keys and mix them however you like, of course you would need a keyboard with detachable key caps for a better use.

Open PowerToys settings and click on keyboard manager menu. First thing first, make sure it’s enabled.

Remap keys

If you want to remap a key, click on the remap keys, then click + button to add a key and then select what key you want that mapped to.

You can also map a single key to a shortcut, for example map V to Ctrl + V.

💡 Just remember that when you map a key to a shortcut, the combination remains even as part of other key presses. So if you’ve mapped V as above, then pressing Ctrl + V now means you’ve pressed Ctrl + V + V.

Remap shortcuts

You can map shortcuts as well, to do so, click on the remap a shortcut button and follow the same procedure. You can click on type button to press multiple keys and at the end you can specify which application this shortcut remap applies to.

PowerRename

PowerRename is another shell extension for balk renaming of files which offers search and replace and supports regex.

Select and right click on multiple files or folders, then click PowerRename. From the opened dialog, select your desired options and once you’re happy, click rename.

PowerToys run

This one is my favourite. It brings back the memories of the days we used to install add-ons on windows XP and 7 to add these sort of functionalities. First, make sure it’s enabled in the settings, and then press Alt + Space. A pop up will open and you can search for your application.

💡 Since I use this shortcut to select my open window to be able to move it between my monitors, I’ve set my shortcut to Ctrl + Alt + Space.

Shortcut guide

Shortcut guide is a quick and easy way to savailable shortcuts for the current state of the desktop. You can open it by pressing and holding down your Win key for 1 second.

Summary

Hopefully these tools will help you as much as they helped me. Have a great weekend ahead and till next article 👋🏻.

Find an item in an array with the new 'findIndex' method 🔎

Mon, 05 Oct 2020 00:00:00 GMT

Searching for items in an array has been the point of discussion for many years and debate on what is the best and optimum way for searching for objects has had many solutions, some effective and some not.

However, with the all new findIndex method on Array.prototype you have the flexibility to search for objects using your own comparison callback method.

The what

The findIndex method returns the index of the first element in an array if the callback method passed to it returns true, otherwise it returns -1.

Copy

const isPerfectSquare = (num) => {
  return num > 0 && Math.sqrt(num) % 1 === 0;
}

console.log([1, 3, 8, 9, 12].findIndex(isPerfectSquare)); // 1
console.log([1, 6, 7, 10, 14].findIndex(isPerfectSquare)); // -1

There are two facts you should know:

This method does not run once it found the first matching element.
It does not change the original array.

Syntax

Copy

array.findIndex(function(currentValue, index, arr), thisValue);

Parameters

function: This is your callback function which checks for a condition to match the element you’re after.
currentValue: This holds the current element’s value.
index: This is an optional parameter which holds the current index.
arr: This is also an optional parameter which holds the array that the current element belongs to.
thisValue: Yet another optional parameter, if a value is passed, it will be used as this value inside the function, otherwise undefined will be passed.

Return value

It will return the index of the found item or -1 if the callback function is not satisfied.

How it works under the hood?

When the findIndex is called with one or two arguments, the following steps are executed:

Let O be ? ToObject(this value)
Let len be ? LengthOfArrayLike(O)
If IsCallback(predicate) is false, throw a TypeError exception
Let k be 0
Repeat, while k < len

Let Pk be ! ToString(k)
Let kValue be ? Get(O, Pk)
Let testResult be ! ToBoolean(? Call(predicate, thisArg, « kValue, k, O »))
If testResult is true, return k
Set k to k + 1

Return -1

Take a selfie 🤳 using Image Capture API and a few lines of code

Sun, 06 Sep 2020 00:00:00 GMT

It’s been a while since I last wrote about Intersection Observer V2, partly because I had a lot on my plate and I was feeling exhausted and partly because I was working on my first PluralSight course on web performance for PWAs.

But I am back, and this time we’re going to review how to take a selfie from your webcam using Image Capture API.

Image Capture API

There are some really useful APIs which allow us to work with media like audio, video, etc. I will write about those too, but this time I wanted to show you how you can extract a frame from your video feed such as your webcam with a few lines of code.

Image Capture API enables us to capture an image or frame from video devices. In addition to capturing data, it also allows you to retrieve information about device capability such as image size, red-eye detection and whether or not there is flash turned on.

Usage

We need to take some actions to be able to work with media devices. First thing first we need to get a reference of the device:

Copy

navigator.mediaDevices.getUserMedia({ video: true })
  .then(mediaStream => {
    // Do something with the stream.
  })

Next we need to get the visual parts of the media stream by calling getVideoTracks method of the mediaStream object:

Copy

const track = mediaStream.getVideoTracks()[0];

Of course we’re assuming that the first item in the array is the one you want to use, but don’t worry if not, you can loop through all the tracks, find the one you need and either get it with its right index, or by calling getTrackById method.

After we’ve got our track, it’s time to capture our image. If you wanted to configure some of the settings on your media device such as zoom level, you need to do it now before capturing our image:

Copy

const capabilities = track.getCapabilities();
// Check whether zoom is supported or not.
if(!capabilities.zoom) {
  return;
}

const zoom = capabilities.zoom.max - capabilities.zoom.min;

track.applyConstraints({ advanced : [{ zoom: zoom }] });

When we’re done with settings, we can then create an instance of the ImageCapture object:

Copy

let imageCapture = new ImageCapture(track);

Once that’s done, you can capture an image from your video feed by calling the takePhoto method:

Copy

imageCapture.takePhoto()
  .then(blob => createImageBitmap(blob))
  .then(imageBitmap => {
    // do something with the photo
  })
  .catch(error => console.error(error));

Permissions

Similar to other APIs you would need permission to be able to access the webcam on devices. If you didn’t see any pop up, make sure you give the camera permission to the site on your browser to be able to get your code working properly. You can try that with below demo.

Demo

You can see a working example of this in my CodePen below:

See the Pen Image Capture API by Yaser Adel Mehraban (@yashints) on CodePen.

I hope you’ve learnt a new thing or two and till next time 👋🏽.

Introducing the Intersection Observer V2 ⛌

Mon, 22 Jun 2020 00:00:00 GMT

As I mentioned in one of my other posts, Intersection Observer will report when an element appears in the viewport, after applying all overflow and CSS clips. However, there is no way currently to find out whether an item is being put on top of this element, or some filter is applied to it which may alter or obscure the element’s display.

Context

Intersection Observer is now widely used by developers especially since Safari added support a while back and it’s now available in all major browsers. If you want to be notified when an element appears in the viewport such as an image, if you want to lazy load it you can use this API. In its most basic form the code looks like this:

Copy

const logIfIsInViewPort = (entries) => {
  for (const entry of entries) {
    if (entry.isIntersecting) {
      console.log(entry);
    }
  }
};

const observer = new IntersectionObserver(logIfIsInViewPort);
observer.observe(document.querySelector('.element-selector'));

The problem

As great as it sounds, this API is not perfect. For example it can’t tell you whether or not an element is covered by another content (which is called occluded), or if a filter is applied to it to hide or obscure it’s content (such as opacity, filter, transform, etc.).

This can lead to dangerous situations where some people might do fraud or abuse the web. So Intersection Observer V2 was born to prevent these types of scenarios.

Intersection Observer V2

As I said, the primary motivation for this API is to prevent those situations where a fraud might be possible, or people could abuse the user’s privacy in a way especially if the said content is put in an iframe.

Intersection Observer V2 works by tracking the actual visibility of the element, just like the end user would see it. By passing an option to it’s constructor, the collection of IntersectionObserverEntry will contain a new boolean called isVisible.

Copy

const handler = (changes) => {
  for(const change in changes) {
    // feature detection
    if(typeof change.isVisible !== 'undefined') {
      if(change.isIntersecting && change.isVisible) {
        console.log(`Visible since ${change.time}`);
      } else {
        // fallback to v1 if not supported
        change.isVisible = true;
      }
    }
  }
}

const observer = new IntersectionObserver(handler, {
  threshold: [1.0],  
  trackVisibility: true, // this will give you the isVisible property
  delay: 100
});

observer.observe(document.querySelector('.target-element-selector'));

Additional parameters

The object which is passed in the constructor has a few properties:

threshold: A list of thresholds which will trigger a callback.
trackVisibility: A boolean indicating whether or not to track the visibility of the element.
delay: A number defining the minimum delay in milliseconds between notification for a given object.

💡 Note that tracking visibility is an expensive operation and should only be used when necessary. The performance implications can cause more harm than good if abused.

How does it determine visibility?

Determining whether the element is visible or not is not as simple as you might think. Based on the spec:

If false is passed as the value of trackVisibility, the element is considered visible.
If the target element has an effective transformation matrix other than 2D then the element is considered invisible.
If the target element or any of its children, has an effective capacity other than 1.0, then it’s considered invisible.
If the target element or any of its children has a filter applied, then it’s considered invisible.
If the browser cannot guarantee the target is fully visible, then it’s considered invisible.

What’s the catch?

This is in a draft state and still work in progress. Apart from Chrome, none of the other major browsers currently support it.

Summary

Although this hasn’t been implemented in most browsers yet, it’s a great tool to prevent some of the common attacks such as clickjacking or likejacking, or cursorjacking.

Chrome - where did save sessions go? 🤔

Wed, 17 Jun 2020 00:00:00 GMT

I remember a while back (not sure how long), Chrome had the ability to save all tabs when you closed the browser regardless of whether it was by accident or not. But they removed that ability and now if you close the window you will lose the tabs.

What now?

Well there are still some ways you can save your current tabs and open them next time you open a new Chrome window.

1. Small hack but works (most times 😁)

Open your task manager and kill the Chrome’s task. It’s a dirty hack but it works. When you open Chrome next time it asks you do you want to restore the last session and when you click the button, it will open all of your previously open tabs.

So first kill the task:

Later when you open a new window:

2. Save them as bookmarks

Yes, you read right. You can save all open tabs as bookmarks and open them all later when you come back. Simply press Ctrl + Shift + d in windows or ⌘ + Shift + d in Mac to save all your open tabs in a new bookmark folder.

Later on when you come back and open a new window, you can right click on the folder you created previously and select open all tabs:

⚠️ This technique won’t help you if your browser crashes for some reason.

3. I am too tired to do all of that

If you don’t want to open your task manager, or don’t want to use bookmarks, don’t worry. There are some extensions which can help you with that.

Tab Session Manager

Tab Session Manager is a Chrome extension which allows you to do:

Save and restore windows and tabs
Manage sessions with name and tags
Auto save when window is closed
Auto save at regular intervals
Import and export sessions
Compatibility with Firefox extension
Cloud sync

It also has Microsoft Edge and Firefox versions which is great.

Session Buddy

Session Buddy is a popular extension with more than 24K installs. It will do all of the above for you automatically in one place. Whether your browser crashes or you choose to close it manually.

Summary

Hope this helps you from a possible disaster if you have millions of tabs open at a time. If you are a tab person, check out my other article on managing your tabs in Chrome here. Until next time, 👋🏽.

Chrome - Arrange your tabs with tab groups 😍

Mon, 15 Jun 2020 00:00:00 GMT

If you’re one of those people with so many tabs open than you can manage, read on.

The why

For those of you who have many tabs open all at the same time and are struggling to keep a tab on your tabs, the time has come to get a bit more organised using this new feature in Chrome called tab groups.

The whole purpose has been to help users keep their tabs organised and easier to find them in their sea of open tabs.

How can I do it?

Simply right click your tab and select Add tab to tab group option. Then there will be a dot right before your tab. Click on the dot, a pop up will open in which you can select a colour for your tab group, and name your group.

If you are clicking on an existing group, you will have the following options in addition:

Add a new tab to the group
Ungroup tabs in a group
Close group
Move the group to a new window
Send feedback if you have any

I’ve demonstrated this in below video:

But I don’t see the option

If you can’t see the option, don’t worry, just go to chrome://flags, then search for tab groups and enable the option. It’s still experimental, but you can use it now.

Summary

Hope this will help you to keep a tab on your tabs and till next time 👋🏽.

Did you know about scroll to text? 📜

Sun, 14 Jun 2020 00:00:00 GMT

You probably have used fragments in links to point a link to a part of your page such as a heading. This technique is used in table of content for example and it’s a common pattern.

Background

To extend the current support for scrolling to an anchor element in a page, the W3C have proposed a solution which basically adds the ability to link to a DOM element with id, or textual content on a page to make it easier for browsers to understand what the users are interested and scroll there when they visit the page. In addition to make it visually easier to follow, they have added the highlight to the text so that users will know where to look at on first sight.

Scroll to Text Fragment

So that’s how Scroll to Text Fragment was born. It has been shipped to Chrome and Microsoft Edge called Scroll to Text Fragment which allows you to link to a specific text on a page, using a fragment provided in the URL. When the page is loaded, the browser highlights the text and scrolls it into view.

This is the generic syntax:

#:~:text=[prefix-,]textStart[,textEnd][,-suffix]
          context  |-------match-----|  context

💡 Square brackets indicate an optional parameter.

Here is an example:

Copy

<a href="https://yashints.dev/#:~:text=Meet&text=Yas">
  Click to go to a specific portion of a page
</a>

And you can test it live by clicking here.

If you want to target an exact text:

Copy

<a href="https://yashints.dev/#:~:text=almond%20croissant%20addict">
  Click to go to a specific portion of a page
</a>

Try it here.

You can also add an end text in which case the text directive refers to a range of text on the page.

Copy

<a href="https://yashints.dev/#:~:text=Although,web%20developer">
  Click to go to a specific portion of a page
</a>

Try it here.

Use cases

Search engines

When search engines will link to pages which contain relevant information to the query, they will scroll and highlight that so the user will find it easier.

Citation / Reference links

Links are sometimes used as citations on some pages where the author wishes to enforce a claim by pointing to a reference such as Wikipedia. These reference pages can contains large chunks of text and finding the information can be difficult. So having the ability to scroll to that section and highlight it is really encouraging to readers to follow the links.

Sometimes you want to share a piece of text to someone for example by email or on social media, it’s really helpful to be able to link directly to the specific section rather than the whole page.

Nowadays people share short snippets directly in tweets or create screenshots which can contain more text and post those. With this they won’t need to do that since links can be more meaningful.

Summary

And that’s how the web is moving towards a better place where users are the centre of attention for us developers and the more we know how to make their lives easier the more they trust us to take care of it for them.

Enjoy your evening and till next time 👋🏽.

Links and JavaScript 🔗

Wed, 03 Jun 2020 00:00:00 GMT

I see some occasions where web developers who are a bit less familiar with accessibility use links incorrectly in different forms. So I thought let’s write a quick guide on Links in JavaScript apps.

Background

It’s safe to assume that anyone who’s used the web have seen them. They take users from one page to another either within the same application or to an external website. There were really hot when they were introduced back in 1966 by a team led by Douglas Engelbart.

In the early 1980s Ben Shneiderman, a computer scientist at the University of Maryland, with his graduate student Dan Ostroff were preparing a videodisk exhibit for a museum. It was then where they used a caption that had all the information and users could click on it to see the photo associated with it.

At first, this discovery was called embedded menu but that term was quickly replaced by hyperlink. Tim Berners-Lee cited Shneiderman’s hyperlink work in his spring 1989 manifesto for what would become the World Wide Web.

Back then, this feature was super hot and it still is. People use it without even thinking about it and it’s become a natural part of the web.

Enough background, tell me what’s this all about

Let’s review what they do:

As mentioned before they allow users to navigate between pages within a site or to an external one.
And more importantly they help search engines and bots to find resources and going from one page to another by following the links. Think of this as a giant maze where they need to blindly follow the links to discover stuff.
And last, the crawlers use them to understand the architecture of a site and the type of content they have. It’s a critical part of their decision making when assigning a page to a specific topic.

But how do you make a link?

`a` tag with `href`

Turns out it’s not as straightforward as one might think. The simplest way to put a link on your site is to use a good old <a> tag with a URL which it points to in the href attribute.

Copy

<a href="/elsewhere">A good link, YES ╰(*°▽°*)╯</a>

`a` tag with `href` and event handler

For those with special circumstances, they might sprinkle a little bit of JavaScript on top and use an event handler to navigate the user programmatically:

Copy

<a href="/elsewhere" onclick="goTo('elsewhere')">Okay ¯\_(ツ)_/¯</a>

This is fine as the link stays functional and you can upgrade the functionality using JavaScript.

`a` tag without `href`

But some people decide to remove the href attribute. This is not a good idea because you now prevented crawlers to find out where this link goes to:

Copy

<a onclick="goTo('elsewhere')">Please don't ╯︿╰</a>

`a` tag with `href` but no link

In other occasions, people use the href attribute without a URL or with a pseudo URL:

Copy

<a href="javascript:goTo('elsewhere')">Seriously? (⊙_⊙)？</a>

This is even worse than the previous case as it just confuses the hell out of every crawler out there.

using `button`

You might think it’s a good idea to use a button to navigate the user, but let me tell you, that’s not a good idea either. The rule of thumb is that if the purpose is to do something on the current page, it could be a button. But if it’s takes users to another page or to an external site, it should be a link.

Copy

<button onclick="goTo('elsewhere')">But why? <( _ _ )> </button>

any other HTML element

It’s worth noting that you shouldn’t simulate a link using any other HTML element and add a click handler with JavaScript:

Copy

<div onclick="goTo('elsewhere')">NOOOOOOOOOOOOOOOO!!!!!!!!!!!! (#`O′)</div>

This not only wrong for crawlers, but also messes up the screen readers and causes a lot of trouble for people with any vision impairment. So please just use a link with a proper URL.

What is a proper URL? I hear you ask

You saw we ruled out the pseudo URLs like javascript:doSomething(), but let’s take a look at URLs more closely.

Copy

http://example.com/old-school-url

example.com/page#section

A URL contains these elements:

A protocol (optional), which defines the protocol to be used for navigation such as http, or https.
It also has a host, example.com or yashints.dev. A host is a name that one or some computers respond to. Usually it points to an IP address.
It has a path to specific asset on that computer such as an HTML file, an image, CSS file or a font. But that’s not always the case, sometimes the path is handled via JavaScript and is used to replace part of the content in a page. This technique is called Single Page Application or SPA.
Some URLs might contain another part such the second example above. That part which comes after the # sign, is called a fragment identifier. By itself a fragment identifier is not a piece of content. It just points to a specific part in the content such as a header.

💡 Because fragments are not suppose to point to different content, crawlers ignore them altogether. This means for single page applications where they use fragment like routing, crawlers don’t follow them and that’s why people use techniques such as server side rendering or SSR to allow crawlers become aware of the page’s content.

Summary

To sum it all up:

Use proper link markup, a tag with href attribute.
Do not use other HTML elements to navigate users.
Do not omit the href attribute on an anchor tag.
And last but not least, do not use fragment identifiers to load different content in SPAs if you can.

Hope you’ve enjoyed reading this and thanks for drooping by. Until next article, Ta.

Microsoft Build 2020 highlights 🔣

Sat, 23 May 2020 00:00:00 GMT

Microsoft Build went on fully online and it came with some really exciting announcements. For us developers some of these can bring massive efficiencies. So let’s see what’s on offer.

WSL2

Windows Subsystem for Linux v2 (aka WSL2) is here with a ton of awesome features allowing developers to have a really great experience working in a side by side Windows/Linux environment.

No Hyper-V image is needed to run docker containers
GPU Support
100% system call compatibility
Increased I/O performance

Linux GUI apps in Windows

This was really an exciting announcements. Microsoft is adding a full Linux Kernel to its Windows Subsystem for Linux v2 (aka WSL2) with GUI support and GPU acceleration.

This will be enabled without us having to use X11 forwarding and it’s mainly designed for developers to run Linux integrated development environments alongside regular Windows apps.

Image from Microsoft’s official blog

Winget, Microsoft’s package manager

This is probably one of those controversial topics where people will say: oh another package manager. But Microsoft believes they have considered all options and this decision was made with lots of considerations.

You can install winget via it’s App Installer package. You can also join their insider program which contains the winget by default.

You can easily add your package to this package manager by following their instructions on their GitHub repository and create a PR containing your package’s information and link to download.

Just so you know how excited people are, there are already 228 open PR’s and 541 closed. Of course not all of those PR’s have been packages, but even so, it’s a lot of packages submitted already.

Microsoft’s new PowerToys Run launcher

Microsoft just released a new Spotlight-like launcher app for Windows 10. This launcher is part of a series of apps which come with the PowerToys tool set.

It’s still preview, but you can install it today. Jump into this page and download the msi file, then install it.

Or if you like to do it a bit more fancy, just run this in your terminal:

Copy

WinGet install powertoys

Remember this requires winget to be installed to work.

Once installed, press Alt + Space and you’ll get this beautiful launcher:

💡 You need to run it manually for the first time for now. This will get fixed once they release their GA version.

Alongside PowerToys Run, Microsoft also released a Keyboard Manager PowerToy which allows you to define shortcuts for your favourite actions, or remap keys like replacing A with B. Check out the other cool tools in their GitHub page.

Project Reunion

Microsoft announced Project Reunion, an effort to bring win32 desktop apps and its Universal Windows Platform (UWP) apps close together. From their page:

Project Reunion makes it easier to build a great Windows app by providing a unified platform for new and existing Win32 and UWP apps. It will unify access to existing Win32 and UWP APIs and make them available decoupled from the OS, via tools like NuGet.

Fluid framework

Microsoft is adding a new type of document to it’s Office suite called Fluid Framework. This allows multiple people to collaborate on a single document with fast speed and offers multiple types of elements like tables, graphs and lists in one place rather than having to use multiple apps for different reasons.

Apart from aforementioned features, it comes with ability to @mention someone to notify them of something, plus activity feeds for each document if you want to be notified of the changes.

And to emphasise their focus in bringing more to open source, they are making this framework open too.

Project Bonsai

Project Bonsai is Microsoft’s effort in creating a platform which lets developers and engineers create autonomous systems. It is a machine teaching service to create and optimize intelligence for industrial control systems. Through machine teaching, subject matter experts without an AI background can break down their expertise into steps and tasks that are imparted to AI agents.

Static Web Apps

Static Web Apps offers static Web applications powered by storage for hosting and Azure Functions as backend APIs. They also have the full pipeline on GitHub using GitHub actions to let you deploy your apps really fast.

These are some of the key features:

Free web hosting for static content like HTML, CSS, JavaScript, and images.
Integrated API support provided by Azure Functions.
First-party GitHub integration where repository changes trigger builds and deployments.
Globally distributed static content, putting content closer to your users.
Free SSL certificates, which are automatically renewed.
Custom domains to provide branded customizations to your app.
Seamless security model with a reverse-proxy when calling APIs, which requires no CORS configuration.
Authentication provider integrations with Azure Active Directory, Facebook, Google, GitHub, and Twitter.
Customizable authorization role definition and assignments.
Back-end routing rules enabling full control over the content and routes you serve.
Generated staging versions powered by pull requests enabling preview versions of your site before publishing.

Image from Microsoft blog

Others

There were dozens of other stuff announced, but here are some of the developer related ones:

Blazer WASM is released
MAUI makes mobile app development easier
Visual Studio is becoming Visual Studio Codespaces
GitHub Codespaces brings amazing developer experience right from your browser
GitHub Discussions for brainstorming feature ideas, help new users
Code scanning and secret scanning on GitHub on private repositories
GitHub Private Instances – a fully managed environment for enterprises
GitHub Actions for Azure are now integrated within: Visual Studio Code, Azure CLI, and the Azure Portal. This makes it easier to deploy from GitHub to Azure!

Hope this has given you some pointers to go play with some of these amazing features and till next time 👋🏽.

Swapping variables in JavaScript 🔁

Fri, 15 May 2020 00:00:00 GMT

There may be many different reasons why you’d want to swap two variables be it just changing two item’s location in an array or when sorting collections. The traditional way is just define a new variable, assign one value to it, put one of the items in the old place, then put the temp variable back in the new place. But my question is, is that the only way?

Traditionally

The old way of swapping two variables is done like below:

Copy

let x = "Yas";
let y = "Hints";

let temp = x;
x = y;
y = temp;

console.log(x); // Hints
console.log(y); // Yas

There is nothing wrong with this approach unless you’re doing it frequently.

Without the temp variable

There is another way you could swap two variables without any temp variable. But this only works with numbers:

Copy

let x = 10;
let y = 20;

x = x + y;
y = x - y
x = x - y;

console.log(x); // 20
console.log(y); // 10

This works two, but now we’re doing three additional operations to save some space, so you need to be careful when you use this one. Another thing to consider with this approach is the chance of having overflows with additions or subtractions (sum should be less than Number.MAX_SAFE_INTEGER which is 9007199254740991).

Bitwise `XOR`

Similar to above approach, you could use XOR to swap the two variables, but this also works only on numbers:

Copy

let x = 3;
let y = 5;

x = x ^ y;
y = x ^ y;
x = x ^ y;

console.log(x); // 5
console.log(y); // 3

If you’re not familiar with XOR, it works on bits. When you perform XOR two bits, it evaluates to 1 if they are different, and evaluates to 0 if they’re the same.

x	y	XOR
0	0	0
1	1	0
0	1	1
1	0	1

So let’s see why this works.

x = x ^ y
y = y ^ x when x = (x ^ y), so the y = (x ^ y) ^ y which equals to x ^ (y ^ y) = x ^ 0 = x. So now our y is the old x.
x = x ^ y when according to our first step x is not x ^ y, and so x = (x ^ y) ^ x = y ^ (x ^ x) = y ^ 0 = y.

Is this better than the previous one, probably faster, but still limited to numbers only.

ES6 destructuring

Destructuring is an ES6 feature which is used a lot in many of the modern frameworks. In its core, it allows you to store array elements in variables.

Copy

let x;
let y;

[x, y] = [1, 2, 3];

console.log(x); // 1
console.log(y); // 2

Now considering how we can use this to swap the elements of an array:

Copy

let x = "Yas";
let y = "Hints";

[x, y] = [y , x];

console.log(x); // Hints
console.log(y); // Yas

This method is much elegant, but still creates two arrays to perform the swapping. So the efficiency might not be that good if you’re swapping many elements.

Summary

Just because a feature is available, doesn’t mean you should use it in every situation. Think about what is the most important factor in the solution you’re implementing. If it’s space, choose one which doesn’t take much, although it’s a bit slower.

If the memory doesn’t matter but speed is important, choose accordingly. But definitely consider the situation before deciding on your approach.

All you need to know about string in JavaScript 🧵

Mon, 11 May 2020 00:00:00 GMT

String is one of the primitive types in JavaScript and we use it in every project we work on no matter what. But how familiar are you with the methods available in JavaScript to work with a string variable? Let’s have a quick look at those in this article.

Introduction

A primitive value such as “Yas” doesn’t have any methods or properties, mainly because it’s not an object. But with JavaScript, methods and properties are available because it treats primitive values as objects.

Let’s have a look at the simplest method you have most definitely used:

String length

The length property returns the length of a string:

Copy

const alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
console.log(alphabet.length); // 26

Pay attention on how length property is available on a primitive type. Not all languages are following the same principle when dealing with such behaviour though. In PHP we have helper functions:

Copy

<?php
echo strlen("ABCDEFGHIJKLMNOPQRSTUVWXYZ");
?>

Finding a text in string

There are a few methods to help you find a sub-string in a String. Let’s go through them and see what are their differences:

`indexOf`

The indexOf method returns the index of the first occurrence of a specified text in a string:

Copy

const txt = "Can you find Yas in 'dryasdust'?";
console.log(txt.indexOf('yas')); // 23

There a few points you need to know here. First, JavaScript counts positions from zero, and second, indexOf is case sensitive.

Copy

const txt = "Can you find Yas in 'dryasdust'?";
console.log(txt.indexOf('Yas')); // 13

This method returns -1 if it can’t find the text:

Copy

const txt = "There is no 0 in 11";
console.log(txt.indexOf('zero')); // -1

You can pass a second argument to let the indexOf know where to start looking for the text:

Copy

const txt = "We have SQL and no-SQL databases!";
console.log(txt.indexOf('SQl', 10)); // 19

`lastIndexOf`

As the name suggests, lastIndexOf is used to find the last occurrence of a text in a string.

Copy

const txt = "Can you find Jam in 'JamStack'?";
console.log(txt.indexOf('Jam')); // 21

This method also returns -1 if it can’t find the text you’re looking for, and takes a second parameter to start the search. However, since this method starts the search backward, the second parameter acts as cutting the string from that position to the end:

Copy

const txt = "Can you find Jam in 'JamStack'?";
console.log(txt.lastIndexOf('Jam', 6)); // -1

`search`

The search method also searches the string for a text and returns the first occurrence of the text:

Copy

const txt = "Can you find Jam in 'JamStack'?";
console.log(txt.search('Jam')); // 13

You might think that search and indexOf are the same. However, there are differences in these two:

search doesn’t accept any other parameter
indexOf cannot take powerful search values such as regular expressions

That’s right, search will accept regex as argument as well, for example, to perform a case insensitive search you might want to use search instead of indexOf:

Copy

const txt = "There is Jam in JamStack!";
console.log(str.search(/jam/i)); // 9

console.log(txt.indexOf('jam')); // -1

You can also search for non usual patterns, e.g. finding any character that is not a word or whitespace:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.search(/[^\w\s]/g)); // 24

`endsWith`

The endsWith methods checks whether the string ends with the specified text. It returns true if it does, and false if it doesn’t:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.endsWith("Stack!")); // true

💡 This method is case sensitive.

`startsWith`

Similar to endsWith, this method checks whether a string starts with the specified text. This method is also case sensitive:

Copy

const txt = "JamStack's got Jam";
console.log(txt.startsWith("JamStack")); // true

`includes`

includes allows you to check whether or not a string contains a specified text and is case sensitive:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.includes("in")); // true
console.log(txt.includes("Jam")); // true
console.log(txt.includes("jam")); // false

`localeCompare`

localeCompare will compare two strings in the current locale. It returns a negative number indicating if the reference string occurs before the compare string, positive if it occurs after, and 0 if they are equivalent:

Copy

const a = 'réservé'; 
const b = 'RESERVE';
console.log(a.localeCompare(b)); // 1
console.log(a.localeCompare(b, 'en', { sensitivity: 'base' })); // 0

Extracting sub-strings

There are three methods which allow you to extract part of a string.

`slice`

slice extracts part of a string and returns the extracted part in a new string. It takes two arguments, start position, and end position (the end position will not be included).

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.slice(9, 12)); // Jam

If you pass a negative value, it will start from the end of the string:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.slice(-16, -13)); // Jam

You can omit the second parameter, and it will extract from start to the end of the string:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.slice(16)); // JamStack!

`substring`

The substring method is similar to slice but it won’t accept negative indexes:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.substring(16)); // JamStack!
console.log(txt.substring(9, 12)); // Jam

`substr`

substr method is similar to slice with one difference that the second parameter is the length of the text to be extracted and not the position:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.substr(9, 3)); // Jam

And if you omit the second parameter, it will extract to the end of the string. Furthermore, if the index you pass is negative, it will count from the end:

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.substr(-9)); // JamStack!

`split`

Although this method is not directly used for extracting a text value, it’s good for splitting the string value by a character and return an array of substrings:

Copy

const txt = "There is Jam in JamStack!";
const words = txt.split(' ');
console.log(words[4]); // JamStack!

A few points regarding this method:

The simplest case is a single character, also referred to as delimiter. For example you can split a tab separated value (TSV) by using str.split("\t").
If the separator contains multiple characters, that entire string needs to be found.
If the separator cannot be found, the return value is an array with one element containing the whole string.
If the separator appears at the beginning or end of the array, it still counts. Meaning the return value is an array with a string value, and one empty string item either at the start or at the end of the array.
If you pass an empty string " as the separator, it splits the string into single UTF-16 characters.

Copy

const txt = "There is Jam in JamStack!";
console.log(txt.split('Jam')); // ["There is ", " in ", "Stack!"]
console.log(txt.split('test')); // ["There is Jam in JamStack!"]
console.log(txt.split('There')); // ["", " is Jam in JamStack!"]
console.log(txt.split('')); // ["T", "h", "e", "r", "e", " ", "i", "s", " ", "J", "a", "m", " ", "i", "n", " ", "J", "a", "m", "S", "t", "a", "c", "k", "!"]

Replacing string content

The replace method, as the name suggests, replaces a part of the string with the provided text:

Copy

const txt = "Who's awesome!";
console.log(txt.replace("Who's", "You're")); // You're awesome!

💡 This method doesn’t change the original string, it returns a new one.

By default, it’s case sensitive and just replaces the first match:

Copy

const txt = "This 🐶 is a good 🐶!";
console.log(txt.replace("This", "That")); // This 🐶 is a good 🐶!
console.log(txt.replace("🐶", "🐕‍🦺")); // This 🐕‍🦺 is a good 🐶!

To do a case insensitive replace or to replace all matches, you could use regex:

Copy

const txt = "This 🐶 is a good 🐶!";
console.log(txt.replace(/THIS/i, "That")); // That 🐶 is a good 🐶!
console.log(txt.replace(/🐶/g, "🐩")); // This 🐩 is a good 🐩!

Case conversion

To convert a string to uppercase or lowercase you can use toUpperCase and toLowerCase respectively:

Copy

const txt = "What's up bro!";
console.log(txt.toLowerCase()); // what's up bro!
console.log(txt.toUpperCase()); // WHAT'S UP BRO!

We also have toLocaleLowerCase and toLocaleUpperCase methods to convert according to user’s current locale:

Copy

const dotted = 'İstanbul';
console.log(`EN-US: ${dotted.toLocaleLowerCase('en-US')}`); // "i̇stanbul"
console.log(`TR: ${dotted.toLocaleLowerCase('tr')}`); // "İSTANBUL"

Concatenation

You can use concat to join two strings together (like + operator):

Copy

let message = "Hello" + " " + "World!";
console.log(message); // Hello World!
message = "Hello".concat(" ", "World!");
console.log(message); // Hello World!

Trimming and padding

`trim`

To remove whitespace from both sides of a string value, you can use the trim function:

Copy

let message = "     Hello World!     ";
console.log(message.trim()); // "Hello World!"

💡 Whitespace in this context is all the whitespace characters (space, tab, no-break space, etc.).

`padStart`

The padStart method adds a given string at the start of the original string (multiple times, if needed), until the resulting string reaches the given length.

Copy

const str1 = '5';
console.log(str1.padStart(6, '0')); // 000005

`padEnd`

The opposite of padStart is the padEnd.

Copy

const txt = 'OMG Jam';
console.log(txt.padEnd(25, '.')); // OMG Jam..................
console.log('OMG Jam'.padEnd(10)); // "OMG Jam   "

Getting the string value

There are two methods where you can get the value of a string in JavaScript. You might say, Yas are you crazy, we already have the value in the variable. But remember that I said JavaScript treats a string variable as an object under the hood, so these methods come from the Object.prototype.

`valueOf`

The valueOf returns the primitive value of an object. For string values, JavaScript does it for you under the hood whenever you invoke a method which needs the primitive value. But you can also call this method to get it:

Copy

const txt = "Yas";
console.log(txt.valueOf()); // "Yas"

`toString`

Similar to the above method, toString is used to return the value of a string.

Copy

const stringObj = new String('Yas');
console.log(stringObj); // String {"Yas"}

console.log(stringObj.toString()); // "Yas"

`repeat`

This method is my personal favourite. You can pass a number to the repeat method and it returns your string repeated by that number. It’s really good if you want to have some long text generated for testing purposes:

Copy

const txt = "Lorem ipsum faked,";
console.log(txt.repeat(5)); // Lorem ipsum faked,Lorem ipsum faked,Lorem ipsum faked,Lorem ipsum faked,Lorem ipsum faked,

Character methods

`charAt`

The charAt returns a new string consisting of the single UTF-16 code unit which is located at the index specified:

Copy

const txt = "There is Jam in JamStack!";
const index = 4;
console.log(`The character at index ${index} is ${txt.charAt(index)}`); // "The character at index 4 is r"

`charCodeAt`

The charCodeAt returns an integer between 0 and 65535 representing the UTF-16 code unit at the given index:

Copy

const txt = "There is Jam in JamStack!";
const index = 4;
console.log(`The character code at index ${index} is ${txt.charCodeAt(index)}`); //The character code at index 4 is 101

`codePointAt`

The codePointAt method returns a non negative integer representing the Unicode point value of the specified index:

Copy

const icons = '☃★♲';
console.log(icons.codePointAt(1)); // "9733" 
'\uD800\uDC00'.codePointAt(0)  // 65536

`normalize`

And last, but not least, the normalize method returns the Unicode normalisation form of a string:

Copy

const myAlias = '\u0059\u0061\u0073\u0068\u0069\u006e\u0074\u0073';
console.log(`${myAlias}`); // Yashints

Summary

Hope you enjoyed reading this and learnt a few tricks which could help you at what you do day to day. And let’s finish this article with a joke:

What do you call it when Wilhelm II makes a string of bad puns? 👉🏽

A Kaiser roll

Let's sort with JavaScript 🔢

Sun, 03 May 2020 00:00:00 GMT

There are many different sorting algorithms out there such as quick sort, merge sort, insertion sort, bubble sort, etc., that could be useful in our day to day life, writing code which gets shipped to production. Knowing all of them is not necessary, but if you have a basic understanding of each one, you can decide on the most efficient one for your scenario.

Introduction

Choosing a suboptimal sort algorithm could lead to longer completion time, code complexity, or worse, a program that crashes half way through an operation.

We use sorting algorithms everyday, Array.sort is one of the sort algorithms which is used to sort an array in ascending order. But that is not a solution for every scenario.

When choosing a sorting algorithm, we need to consider the complexity, or the number of operations performed (usually mentioned as O(x), which is read Big O of x) and also number of swaps along the way. So let’s review and implement some of the most used ones together and learn about their complexity.

Bubble sort

The way bubble sort works is very simple. You compare the first item of the collection with the second. If the first one is bigger, then swap the two. If not, move to the second item, and repeat the same. We keep repeating this until we reach the end of the list.

So far we have bubbled up the biggest item of the list to the far right side in its position. Now we repeat this for the rest of the items again until the list is sorted.

Let’s see this in action:

Copy

function bubbleSort(list) {
  let len = list.length;

  for(let i = len - 1; i >= 0; i--) {
    for(let j = 1; j <= i; j++) {
      if(list[j - 1] > list[j]) {
        let temp = list[j - 1];
        list[j - 1] = list[j];
        list[j] = temp;
      }    
    }  
  }

  return list;
}

bubbleSort([7, 5, 2, 3, 9, 6]); // [2, 3, 5, 6, 7, 9]

As you can see this algorithm is not optimal, in fact it is one of the heaviest in terms of number of operations in worst-case scenarios. But in terms of swaps, it’s one of the best since it sorts in place.

The complexity of bubble sort in worst-case is O(n²), read as big O of n square, where n is the number of items in the collection.

However, in a best case scenario (already sorted collections), it will be O(n) with O(1) swaps.

Case	Complexity
Worst-case performance	O(n²) comparisons
	O(n²) swaps
Best-case performance	O(n) comparisons
	O(1) swaps
Average-case performance	O(n²) comparisons
	O(n²) swaps

Selection sort

Selection sort is really simple like bubble sort. We go through the list, find the index of the lowest element, then swap the lowest element with the first one. Now that the first item is sorted, we repeat this for all remaining elements.

Let’s see this in action:

Copy

function selectionSort(list) {
  let minIndex, temp,
      len = list.length;
  
  for(let i = 0; i < len; i++) {
    minIndex = i;
    for(let j = i+1; j < len; j++) {
      if(list[j] < list[minIndex]) {
        minIndex = j;
      }
    }

    temp = list[i];
    list[i] = list[minIndex];
    list[minIndex] = temp;
  }

  return list;
}

selectionSort([11, 25, 12, 22, 64]); //[11, 12, 22, 25, 64]

Let’s see how the list is sorted in each iteration in the above example:

Sorted list	Unsorted sublist	Lowest elements
[]	[11, 25, 12, 22, 64]	11
[11]	[25, 12, 22, 64]	12
[11, 12]	[25, 22, 64]	22
[11, 12, 22]	[25, 64]	25
[11, 12, 22, 25]	[64]	64
[11, 12, 22, 25, 64]	[]

In terms of complexity, this algorithm remains the same regardless of which scenario we’re facing. Which is O(n²) for comparisons, and O(n) swaps. But if you look at the code, it’s self explanatory and simple and sometimes we just want exactly that. In terms or swaps, it’s less than bubble sort.

Case	Complexity
Worst-case performance	O(n²) comparisons
	O(n) swaps
Best-case performance	O(n²) comparisons
	O(n) swaps
Average-case performance	O(n²) comparisons
	O(n) swaps

Insertion sort

This is like when I play cards and someone is handing me the them one by one. I usually put them in my hand in order as I receive them. Insertion sort builds the final list one item at a time. This means it’s less efficient for large lists relative to it’s competitors such as quick sort, or merge sort.

However, it provides several advantages:

Simple implementation (we’ll get there shortly).
Efficient for small data sets.
More efficient than bubble or selection sorts.
Adaptive, i.e. efficient for already sorted collections.
In place.
Online, can sort a list as it receives it.

Let’s see how it works in action:

Copy

function insertionSort(list){
  let i, len = list.length, item, j;

  for(i = 1; i < len; i++){
    item = list[i];
    j = i;

    while(j > 0 && list[j-1] > item) {
      list[j] = list[j-1];
      j--;
   }

   list[j] = item;
  }

  return list;
}

In terms of complexity, it’s similar to bubble sort in worst and average cases with O(n²) for both comparisons and swaps. But in best case, it’s really efficient with O(n) comparisons and O(1) swaps.

Case	Complexity
Worst-case performance	O(n²) comparisons
	O(n²) swaps
Best-case performance	O(n) comparisons
	O(1) swaps
Average-case performance	O(n²) comparisons
	O(n²) swaps

Merge sort

Merge sort is in the divide and conquer algorithms and is implemented with a recursive pattern. We break down the list into small pieces until you have one item in each piece. Then we merge them back together but will compare them and put the items in order.

It’s really easy to understand, but let’s see it in action:

Copy

function mergeSort(list) {
   let len = list.length;
   if(len < 2)
      return list;
   let mid = Math.floor(len/2),
       left = list.slice(0,mid),
       right =list.slice(mid);
   
   return merge(mergeSort(left),mergeSort(right));
}

function merge(left, right) {
  let result = [],
      lLen = left.length,
      rLen = right.length,
      l = 0,
      r = 0;
  while(l < lLen && r < rLen) {
     if(left[l] < right[r]) {
       result.push(left[l++]);
     }
     else{
       result.push(right[r++]);
    }
  }  
  
  return result.concat(left.slice(l)).concat(right.slice(r));
}

Merge sort is much better in terms of complexity from previous algorithms. It takes O(n log n) operations to sort an array. In terms of memory needed, it’s O(n) total with O(n) auxiliary if we use array and O(1) if we use a linked list.

Case	Complexity
Worst-case performance	O(n log n)
Best-case performance	O(n log n)
Average-case performance	O(n log n)
Worst-case space	O(n) total, O(n) auxiliary with list, O(1) with linked list

Quick sort

Quick sort is similar to merge sort, with the difference that the we don’t split the collection in half. We choose a pivot point and split from there. Once we have chosen the pivot point, we put all smaller items to the left and all the larger items to the right of that.

That means that the pivot point itself is sorted now. We continue this for left and right side recursively until we have the full list sorted.

Choosing the pivot could be random, middle point, first, or last item of the list. There are many ways to do this each with their own pros and cons.

Let’s see this in action to better understand the difference:

Copy

function quickSort(list, left, right) {
   let len = list.length, 
   pivot,
   partitionIndex;


  if(left < right) {
    pivot = right;
    partitionIndex = partition(list, pivot, left, right);
    
   //sort left and right
   quickSort(list, left, partitionIndex - 1);
   quickSort(list, partitionIndex + 1, right);
  }
  return list;
}

function partition(list, pivot, left, right) {
   let pivotValue = list[pivot],
       partitionIndex = left;

   for(let i = left; i < right; i++) {
    if(list[i] < pivotValue) {
      swap(list, i, partitionIndex);
      partitionIndex++;
    }
  }
  swap(list, right, partitionIndex);
  return partitionIndex;
}

function swap(list, i, j) {
   let temp = list[i];
   list[i] = list[j];
   list[j] = temp;
}

quickSort([11,8,14,3,6,2,7],0,6); 
//[2, 3, 6, 7, 8, 11, 14]

As you can see, the more efficient the algorithm gets, the more complex the implementation will be. In terms of complexity, it’s worst than merge sort in worst-case, and equal in average and best.

Case	Complexity
Worst-case performance	O(n²)
Best-case performance	O(n log n) with simple partition, O(n) with three-way partition
Average-case performance	O(n log n)
Worst-case space	O(n) auxiliary

Heap sort

Heap sort is a comparison based sort, you can think of it as an improved version of selection sort. It divides it’s input into a sorted and an unsorted region, then iteratively shrinks the unsorted region by extracting the largest item and inserting it into the sorted region.

The unsorted region is kept in heap data structure to more quickly find the largest item in each step.

💡 Since there is no heap data structure in JavaScript, we can use an array to represent it.

That was a mouthful, so let’s see it in action:

Copy

function heapSort(list) {
  let len = list.length;
  let i = Math.floor(len / 2 - 1);
  let j = len - 1;

  while(i >= 0) {
    heapify(list, len, i);

    i--;
  }

  while(k >= 0) {
    [list[0], list[k]] = [list[k], list[0]];

    heapify(list, k, 0);

    k--;
  }

  return list;
}

function heapify(list, len, i){   
  let largest = i;
  let left = i * 2 + 1;
  let right = left + 1;

  if(left < len && > list[left] > list[largest]) {
    largest = left;
  }

  if(right < len && list[right] > list[largest]) {
    largest = right;
  }

  if(largest != i) {
    [list[i], list[largest]] = [list[largest], list[i]];

    heapify(list, len, largest);
  }

  return list;
}

In the above code snippet, heapify function compares three elements, the parent, and two children. It then makes sure that they are in the correct order for a max heap since we’re building the heap from bottom up.

Case	Complexity
Worst-case performance	O(n log n)
Best-case performance	O(n log n) distinct keys, O(n) with equal keys
Average-case performance	O(n log n)
Worst-case space	O(n) total, O(1) auxiliary

Summary

You should have a good understanding of these sort algorithms by now. If not, I recommend going through them again and try to write a few examples with a pen and paper. Don’t worry if you have trouble understanding the more complex ones like heap sort. It’s completely OK as I had the same trouble initially. But with practice and trying to implement them I learnt them at the end.

There are many other sort algorithms out there, so feel free to explore them and compare the way they work with what you’ve learnt so far.

Thanks for reading and enjoy sorting your collections.

⚡ All the images in this post are pulled from the algorithm’s Wikipedia pages.

Have you ever thought about for loops? ➰

Fri, 24 Apr 2020 00:00:00 GMT

Using a loop is almost a must in our day to day life. But have you ever thought what kind of loop should you use? Do you know the difference between enumerables and iterables? This article sheds some light in this space, so read on if you’re interested.

Background

Looping has seen quite a few changes from when I started programming. I remember using while loops and thinking wow, this is cool (I was printing starts on console using MS-DOS).

For loop is another way to iterate through a data structure which contains many items. But non of these methods iterate over the actual structure. They use a sequence, usually named i which mirrors the identifier for you.

Copy

const fruits = ['🍉', '🍎', '🍌'];

for (let i = 0; i < fruits.length; i++) {
  console.log(fruits[i]);
}

// 🍉, 🍎, 🍌

Here i is an index which allows you to access the elements of fruits array and is not part of the array itself. But with progress on JavaScript towards more modern ways of programming, things have changes now. We have for...in, and for...of loops which use a different mechanism for going through items in a collection.

Concepts

Before we delve into these newer ways of iteration, we need to be on the same page around some concepts. So let’s go through them now:

Iterables: This is a kind of loop where we are performing a repetition. Meaning the same set of actions are performed on each item.
Enumerables: This is a kind of loop where we making mention of, or counting items one at a time. This means the collection’s elements can be distinctly identified and referenced.

That didn’t click for me at first, but after going through some examples, it finally made sense. If you consider a full pencil case, that’s an enumerable. When you line up at Aldi to pay, that line is an iterable. A wad of cash is an enumerable, whereas a row of keys on your keyboard is an iterable.

💡 In order for an object to be iterable, it MUST implement an @@iterator property which returns an iterator. An iterator object is one which has a next method which returns the next item in the collection. That’s why an object is not an iterable.

By now you should have started to see the pattern here. The best analogy I came across is:

😍 Enumerables are like rectangles whereas iterables are squares. This means all iterables are enumerables, however, not all enumerables are iterables.

`for...in`

So let’s start from enumerables. You can go through enumerables using for...in. The use case is mainly to go through key-value pairs in an object:

Copy

const superCar = {
  make: 'Lamborghini',
  model: 'Veneno',
  year: '2020'
};

for (let key in superCar) {
  console.log(`${key} --> ${superCar[key]}`);
}

// make --> Lamborghini
// model --> Veneno
// year --> 2020

You can also use for...in to go through index values of an iterable like an array or a string:

Copy

let fact = 'Lamborghini is the best!';

for (let index in fact) {
  console.log(`Index of ${fact[index]}: ${index}`);
}

// Index of L: 0
// Index of a: 1
// Index of m: 2
// Index of b: 3
// ...

💡 Beware that you can’t iterate on Symbol properties with for...in and that’s because Symbols are primitive data types that are always unique. They are generally used as private properties to avoid name clashes when inheritance is used.

Using `for...of`

This kind of loop is applicable to “iterable objects” meaning the item after of MUST be an iterable:

Copy

const fruits = ['🍉', '🍎', '🍌'];

for (let fruit of fruits) {
  console.log(`${fruit} is delicious.`);
}

// 🍉 is delicious.
// 🍎 is delicious.
// 🍌 is delicious.

And again we can use it on strings, but with a slight difference:

Copy

let fact = 'Yas';

for (let char of fact) {
  console.log(char);
}

// Y
// a
// s

Summary

We saw the difference between these two modern ways of looping through collections, let’s make more informed decisions as to use what where and write cleaner, more readable code 👊🏽.

Stay on top of performance with Lighthouse CI 🚦

Mon, 13 Apr 2020 00:00:00 GMT

You’ve spent many hours trying to improve your web performance and have got it to a good speed. What happens next? How do you ensure it remains in good shape especially if you’re working in a team with diverse backgrounds and level of coding.

Web performance must be monitored

I truly believe web performance shouldn’t be an item in your backlog, rather part of acceptance criteria on each story. That said, having discipline is hard when it comes to web performance. Each member of the team should be aware of dos and don’ts so that they can right code which doesn’t add a few seconds to current page load time, or user’s interactivity with the page.

In order to do so, you need to constantly monitor the performance of your application and when is a better time to do it than your CI/CD pipeline or when creating a pull request to get the code merged in (it’s worth mentioning having a CI/CD pipeline is not necessary to have a high quality, well written and performing site).

In addition to all of that, your site performs differently from your localhost, so you need to ensure you’re testing close to your actual environment.

Lighthouse CI

Lighthouse CI is a set of commands that make continuously running, asserting, saving, and retrieving Lighthouse results as easy as possible.

This is a great tool which allows you to run audits on your code base using command line, and raise an alert when someone unintentionally downgrades the performance.

What to measure?

There are countless metrics on which we could focus on. Some are user centric like time to interact, some are resource centric such as the size of your JavaScript or CSS files. Lighthouse CI allows us to easily focus on the size of the files. You can create two types of budgets:

1- Number of given resource: This will help you to for example enforce having only 3 JavaScript and 1 CSS file. 2- Size of a given resource: This budget will enforce that a given resource is of certain size, e.g. only 500kb.

How does it know?

You’re probably wondering how does Lighthouse CI know what to measure and when to raise an alert. This **||is done via a JSON file which is usually named budget.json, but you can choose any name you prefer.

Performance budget

The content of this file is an array of objects representing the budgets for a path. The content will look like below if you only have one path to assert:

Copy

[
  {
    "resourceSizes": [
      ...
    ],
    "resourceCounts": [
      ...
    ]
  }
]

Within the resourceSizes and resourceCounts arrays, you can add individual objects that specify the resourceType and its budget. The budget can be any positive integer and the resourceType can be any of the following:

Resource type	Description
script	JavaScript files
document	HTML documents
image	Images
stylesheet	CSS files
font	Webfonts
media	Other media
other	Any other resource which doesn’t match the above, e.g. data transfers over Websocket connections
third-party	All resources from a third-party domain
total	All resources combined

For example, the following file will set a 200Kb budget for all scripts, a 500Kb for all images, and a 500Kb for all other resources. Then it sets the total network requests to be 25.

Copy

[
  {
    "resourceSizes": [
      {
        "resourceType": "script",
        "budget": 200
      },
      {
        "resourceType": "image",
        "budget": 500
      },
      {
        "resourceType": "total",
        "budget": 500
      }
    ],
    "resourceCounts": [
      {
        "resourceType": "total",
        "budget": 25
      }
    ]
  }
]

As of Lighthouse 5.3 you can add a path property to select which pages the budget should apply to. This is very handy because it allows you to have a centralised budget file for all your pages.

Copy

[
  {
    "path":"/*",
    "resourceSizes": [
      ...
    ],
    "resourceCounts": [
      ...
    ]
  },
  {
    "path":"/blog",
    "resourceSizes": [
      ...
    ],
    "resourceCounts": [
      ...
    ]
  },
  {
    "path":"/contact",
    "resourceSizes": [
      ...
    ],
    "resourceCounts": [
      ...
    ]
  }
]

How should you choose your numbers?

Finding the performance budget is not an easy task since it will depend on your application’s structure and bundling style. However, you can use this useful tool called Performance Budget Calculator to have a starting point. You might have to run the tool multiple time to each to a point where you’d be happy with the result. You don’t want to be too aggressive and also too complacent either.

Using the CI locally

You can install the CI locally with:

Copy

npm install -g @lhci/cli

It also has a dashboard which shows you the historical runs, compares reports and store their results.

Copy

npm install @lhci/server

Before running the CI, you need to have a configuration file to tell the tool what to test and where to find the budget file (you could also pass those individually without a config file, but it’d be a pain):

Copy

// lighthouserc.json
{
  "ci": {
    "collect": {
      "numberOfRuns": 2,
      "startServerCommand": "yarn serve",
      "url": "http://localhost:8080"
    },
    "preset": "lighthouse:recommended"
  }
}

Then you can run the tool on your local web server by first running:

Copy

lhci autorun --config=./lighthouserc.json

and then:

Copy

lhci assert --no-lighthouserc --budgetsFile=budget.json

And you will get a result similar to below based on your app’s performance:

Copy

2 result(s) for http://localhost:8080/

  ×  resource-summary.script.size failure for maxNumericValue assertion
     Keep request counts low and transfer sizes small
     Documentation: https://developers.google.com/web/tools/lighthouse/audits/budgets

        expected: <=102400
           found: 2416174
      all values: 2416174, 2416174


  ×  resource-summary.total.size failure for maxNumericValue assertion
     Keep request counts low and transfer sizes small
     Documentation: https://developers.google.com/web/tools/lighthouse/audits/budgets

        expected: <=512000
           found: 2437694
      all values: 2437694, 2437694

Assertion failed. Exiting with status code 1.

Using the tool on your PRs

I am focusing on GitHub using actions here. But you can apply this technique to Azure DevOps, CircleCI, TeamCity, or any other tool you’re using. For this to work you will need to have a GitHub flow in your git repository defined.

This article’s focus is not to teach you how to create a workflow, but here is the bare minimum you will need to have. Create a .yaml/.yml file in the .github/workflows folder, you can name it whatever you want. I usually name them after what’s their purpose, e.g. lighthouse.yml or lhci-performance-bot.yml.

But before we get to the content of our yaml file, let’s see what we need to do on each PR:

Pull down the code
Build the app
Deploy to a temporary site (I am using Netlify Preview here, but you can use others like staging slots on Azure App Service, etc)
Run Lighthouse CI on the site
Fail if the conditions are not met

For this to work, we need an application. So I just spun up a new Vue application using Vue CLI, then added my workflow to it and pushed it to a new GitHub repo.

Here is how my workflow looks like:

Copy

name: Lighthouse CI

on:
  pull_request:
    branches: [master]

jobs:
  lighthouse:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Setup Node 12
        uses: actions/setup-node@v1
        with:
          node-version: 12.x
      - name: Yarn Install and Build
        run: |
          yarn
          yarn build
      
      - name: Waiting for 200 from the Netlify Preview
        uses: jakepartusch/wait-for-netlify-action@v1
        id: waitFor200
        with:
          site_name: "lighthouse-ci-pr"

      - name: Audit URLs using Lighthouse
        uses: treosh/lighthouse-ci-action@v2
        with:
          urls: |
            $DEPLOY_URL/
            $DEPLOY_URL/about
          runs: 3
          budgetPath: ./budget.json
        env:
          DEPLOY_URL: ${{ steps.waitFor200.outputs.url }}

What is happening is that I am just creating an action on each PR. In this action, we need to pull down the code, install node, run yarn and yarn build to build the application, wait for Netlify to deploy our PR into a preview site, then get the URL and inject it into the next step.

I’m just using wait-for-netlify-action, but it’s not that hard to implement this part. Then we will be using the lighthouse-ci-action to run our audit on the paths we have specified.

Notice I am not using a Lighthouse configuration file in this workflow because it’s not needed. You can pass the parameters directly to the action like I’ve done above. I am just auditing the home and about pages since I’ve only got these two pages.

You can pass as many URL’s to this as you like, but if there are many of the, it makes more sense to add the config file back.

💡 Remember to setup Netlify for your site and set it to create a deployment for every PR. Then get the site id and pass it to the wait-for-netlify-action action.

Pro tip: When you setup your Vue CLI generated application which is using Vue Router, the non default routes won’t work if the user types them in the browser URL bar and hits enter. That’s because that route is handled on the client side and server doesn’t know about it. To fix that you need to add a URL rewrite rule to return your index.html for all legitimate routes.

In case of Netlify, just create a file named _redirects inside your public and add the following line to it.

Copy

/*    /index.html   200

Result

I created a branch and pushed it up. Then created a PR and voila, the action kicked in and failed, since I have a large image on the about page.

I then reduced the size of the image and pushed the changes up. This time it passed.

Summary

We learned together how to use Lighthouse CI, a tool which allows you to run audits on your sites from command line in our CI/CD pipeline to make sure no one is downgrading the performance when they work on the application.

This way, we’re always on top of our game rather than creating backlog items to find the issues and fix them later.

In addition to that, your users are going to be happy and remain happy with their experience which is worth all the effort.

Hope you’ve enjoyed reading this and happy performance enforcement 👋🏽.

How I recovered a day's worth of work with git 🏥

Sun, 05 Apr 2020 00:00:00 GMT

I was working on a project a week ago and had a local branch (with a few commits) I was going to push to upstream to create a PR (pull request). However, after pulling down master and rebasing, I realised all of my code except my first commit is gone after rebase 😱.

Background

I had a local branch which I was working on. Since I’d made a few changes before end of the day, I committed those and went about to do some parenting.

The day after that, I restarted my work and had many other changes in a few other commits by the time I was ready to push my code to upstream. But before we do it, it’s our practice to pull down master and rebase in case there is any conflict, or changes made by someone else has had any side effects on the bit we’ve done.

Rebase

So I pulled down the master branch and rebased it directly from origin using (don’t do this at home please, I’ll explain later):

Copy

git pull origin master --rebase

Once done, I realised I had some conflicts since someone else had worked on one of the files I’d modified. So I started the merge conflict process and made sure all was good. However, I didn’t realised that when I tested the web application, the latest code wasn’t refreshed, and I was seeing my own code (don’t ask me why, it’s an old code base and you need to rebuild and hard refresh every time you make a change).

Having seen all is good I created a PR and went for lunch. One of colleagues then reviewed the PR and since all my changes had gone apart from the small parts from yesterday’s commit, he approved and completed the PR.

Someone raised a bug

I was working on something else when I realised someone has raised a bug on the feature I’d implemented and when I checked the code I was in total shock since most of my code wasn’t there anymore. So I checked my local branch and saw that’s the case, in fact several of my commits were missing from that branch altogether.

Later I realised that I’d forgotten to add everything before committing 🤦🏽‍♂️. Just to clarify this was during a really crazy time where we were working day and night for over a week to deliver this project for all of our hospitals across Australia and New Zealand to monitor their resources in intensive care like number of available beds, ventilator, etc. I will tell you all about it once I find some time.

Initial try

I started by running:

Copy

git reflog

I saw anything but my commits which expected because that commit command never had happened due to the files that weren’t added.

Checking stash

Next thing I checked was my stashed files:

Copy

git stash list

This showed me there were two sets of stashed files.

Copy

stash@{0}: WIP on elmah: 82eb0a4 Merged PR 426: Fixed pass change page
stash@{1}: WIP on availability: 87b2264 Dashboard is now grouping based on hospital category, has got a filter and a search box to search for hospitals

Then I looked into each of these with git stash show stash@{0} and git stash show stash@{1} but I didn’t see any of those.

checking git database

At this point I realised that I needed some help and I contacted Scott one of our git gurus to help me fine out what happened. He suggested we start by checking the database for any lost commit using:

Copy

git fsck --lost-found

From the documentation:

—lost-found:

Write dangling objects into .git/lost-found/commit/ or .git/lost-found/other/, depending on type. If the object is a blob, the contents are written into the file, rather than its object name.

This showed me a bunch of stuff including some dangling commits.

Copy

Checking object directories: 100% (256/256), done.
Checking objects: 100% (4282/4282), done.
dangling tree 484106954fff79e5a4d7b8fd53fca05311e56e79
dangling blob d10124a92b82ccdcb1521ef814f815b75223e0f3
dangling commit 4c8288a8c30d4171c4935982e18ebd842a2c2811
dangling tree ae02e6d9b291775c9bc1eb1489ba341f8b033f48
dangling commit 2ec37c71f59766d55c09830a5b6e2c5e1c741a95
dangling blob 40c3c7ccbd197c78ceddf71c2be01b0db2114ba9
dangling tag a70363ef35f023e57e00fc625bee362c2e9e4bc9
dangling commit 1184e4a583e3d0f9cf9f3cb994a3375b16d122cd

So we started to go through some of these to find out which one contained the files I was after.

We checked out a few of those commits and couldn’t find what we were after. Then he suggested we check those dangling blobs which technically are a tree object containing a snapshot of everything in your current branch.

Checking git dangling blobs

You cannot checkout those objects, instead you need to use below command where the last part is the hash code of the blob you’re looking into:

Copy

 git ls-tree -r d134e9e

You can copy this from the output of the previous command and you don’t need the whole hash value, just a few from the beginning.

The output of this command looks like this:

Copy

100644 blob ea25f3a9fe6094d700588f33c340d32281a7614b    src/ARV.Database/ARV.Database.sqlproj.orig
100644 blob 2e47fe1d11bd2624065866591af00f80f16e5e1e    src/ARV.Web.Reach/Partials/hird/incident.form.campuses.list.html
100644 blob 2a6b2547ace20f6bbda788dc4e97e4a749281185    src/ARV.Web.Reach/Scripts/app/core/filters.js
100644 blob a366e78d88695f3c1a2b5617ec03fd24387f2323    src/ARV.Web.Reach/Scripts/app/hird/incident.form.controller.js
100644 blob 461c0e11c1ad64a0bee7bb7f57ba9a99314b1510    src/Arv.Cases.External/App_Data/Arv.Cases.External.XML
100644 blob 276ace5a9dae6be5613beebedcfbf5a64f8e4c4b    src/Arv.Services/Dtos/ReferenceDataDto.cs.orig
100644 blob 5336a10e14295222f28ef087643ac6696dc3cb73    src/Arv.Services/Reach/IIncidents.cs
100644 blob 6e186554f4de5347a754d39ebca1dd8867b7eb0a    src/Arv.Services/Reach/Incidents.cs
100644 blob cf70ac7b4af8f9d8e89c2d84e45a23bdaf7c99de    src/Arv.Services/ReferenceData.cs.orig
100644 blob 502e1286886624c6e0d46b9f65886fbc38b86bca    src/Arv.Web.Core/Controllers/Reach/IncidentController.cs
100644 blob 5ef3891af8afa82dd7cdfc45058a6186e4656c34    src/Arv.Web.Core/Controllers/ReferenceController.cs
100644 blob 49e5d63ba2e59c6389c21c24b81809326711fd27    src/Arv.Web.Core/Controllers/ReferenceController.cs.orig

After checking a few of these blobs, I finally found the one which contained a version of the file close to the finishing state (yes I had to redo some of my work, but now I had to spend half an hour instead of a day).

Checking the file

Once we found the blob, we had to check what’s the content of those files. You can do that using:

Copy

git cat-file -p cf70ac7b

Now you can see the content printed out to the console. But if the file is bit enough like my situation, you could just pipe this command to write it to a file like so:

Copy

git cat-file -p cf70ac7b | out-file temp.js

If you’re using linux or Mac, just use your version of the equivalent.

And that’s it, repeat this until you have all of the files you need.

Summary

A few lessons I learnt from this accident:

Always check you have added all your files before committing (I know it sounds obvious, but it could happen to you the way it happened to me).
Always make sure you commit your changes often after each set of critical change.
This one has become my preference now. Rebase from your local branch and not upstream:

Copy

git add .

git commit -m "Your awesome and informative commit message"

git checkout master

git pull

git checkout feature-branch

git rebase master

And if you lost your code, don’t give up, git is a really powerful tool.

Hope this helps someone to save some time in case of a similar accident.

Stop using `console.log`, start using your DevTools 🛠️

Thu, 19 Mar 2020 00:00:00 GMT

There are many situations where we want to see what’s wrong with out code without the trouble of changing source code and push the changes again, regardless of whether we’re using local environment or production. Most folks start by writing console.log statements throughout their code base and go step by step to find the place where the bug is happening. That’s OK for beginners and where you have access to source code, but what if you don’t want to waste so much time or you don’t even have access to source code?

DevTools

All major browsers have developer tools (aka DevTools) nowadays. It’s important for us web developers to know them well not just because we use them as our daily drivers to test our applications and see how they look like once deployed, but also for times when something is wrong as we want to find them efficiently.

That’s when the DevTools comes very handy if you know the ropes. DevTools have a lot of functionalities; refer to my articles Chrome DevTools can do that? and FireFox DevTools can do that? to have a glimpse of some of these. However, in this article we only focus on debugging experience.

The bug

In order to be able to have a common ground, we need a bug to fix. I’ve created this demo which will be working on in Glitch. In this demo, you would select a date and press the calculate age button. It will calculate your age and show it in the bottom section.

However, there is a bug where sometimes it calculates the age by one year higher than actual. We will be looking into how to fix this.

DevTools’s debugger

Both Chrome and Firefox have a debugger section we will be using in this article. I won’t go through Edge because it’s the same as Chrome.

Chrome

You can open your DevTools in Chrome using Ctrl+Shift+J on Windows or Ctrl+Option+J.

Once opened, navigate to the Sources tab. You will see a file navigator pane on the left where you can inspect the files that the page uses. Then there is the editor pane in the middle where you can click on the file from the navigator pane and see the content and edit them locally in the browser. And last you will see the JavaScript debugging pane where you have a set of features which you’ll learn about shortly.

💡 If you resize the DevTools you might see these panes in different positions.

Firefox

In Firefox you need to use Ctrl+Shift+I on Windows or Ctrl+Option+I to open up the web developer tools.

When opened, click on debugger tab which is very similar to what you saw before. On the left, you’ll see the navigation pane, next to it is editor pane and to the left (or below depending on how wide you’ve got your DevTools open) is the debug pane.

Setting a breakpoint

When it comes to debugging, a common method is to use console.log statements throughout the code base which sometimes is abused.

Copy

const yo = document.querySelector('#yo');

function getAge() {
    console.log('Getting the date of birth value');
    const dateString = document.querySelector('#age').value;
    console.log(`date of birth is ${dateString}`);
    var today = new Date();
    var birthDate = new Date(dateString);
    var age = today.getFullYear() - birthDate.getFullYear();
    console.log(`age is ${age}`);
    var m = today.getMonth() - birthDate.getMonth();
    console.log(`Birth month is ${m}`);
    if (m < 0 || (m = 0 && today.getDate() < birthDate.getDate())) {
      console.log('The birth month is negative or is zero, we need to reduce the age by one year');
      age--;
      console.log(`Real age is ${age}`);
    }
   yo.textContent = age;
}

And then you look at the console to see where the bug might be. But this process is painstakingly slow which impacts your productivity a lot. So let’s see how breakpoints help us get to the point real quick.

Breakpoints have the advantage of being in real time compared to console.log statements where you need to wait for the code to get deployed and inspect the console. Apart from that, with console.log statements, you need to explicitly mention which values you want to expect whereas in a breakpoint debugger shows you all of them.

Now let’s take one step back and see how we can go about finding where to set our breakpoint to. In some cases like this you will think OK, the age is correct sometimes and not others depending on the month. So you can find your file and set your breakpoint right where the if condition is.

In some other cases where the code base might be larger, or you might be new to the team, it makes sense to follow the flow of the page. In our case user selects a date and then clicks on the calculate age button.

The logic behind happens under that click, you the best thing for us is to set a breakpoint on the click event listener. Event Listener Breakpoints are designed for this.

Chrome

In Chrome, in the debug page, click on the Event Listener Breakpoints and expand the section. Then find the Mouse category and select the click event.

Now click on the calculate age button and DevTools will pause when the event listener executes. You might need to click on the Resume Script Execution if you’re using a platform like Glitch, but for your own app that won’t be needed. When the breakpoint reaches the button in your HTML, press Step into function and you’ll end up in our getAge function. Once you’ve stepped into the function you wanted, you can go one line at a time by clicking on the step over button or pressing F10.

Firefox

You can use the same approach to enable a mouse click event listener breakpoint in Firefox, simply expand the Event Listener Breakpoints section in the debugger pane, then expand Mouse and select the click event.

Once enables, you could follow the same procedure we described before. First click on the calculate age button and the debugger automatically pauses on the first function call. In case of using a platform like Glitch or CodePen, that might not be what you’re after, so simply press the Resume button until you see the breakpoint stop at the getAge function. Then click on the Step In button or press F11 to go inside the function. Once you’re inside the function simply press Step over button or F11 to go line by line from thereon.

Line breakpoint

Line breakpoints are mainly used when you’ve narrowed down where the bug might actually be. In our case when we stepped through the getAge function, we saw that the age is calculated based on the year, then there is an if condition which will be responsible to reduce the age by one if the month is less than the current month minus the month of birth.

So we roughly know that if the age is calculated right in some cases and wrong by one year in others, the if condition is where we should set our line breakpoint on.

There are two ways to do this in DevTools, one is to follow the event listener breakpoint flow explained above. But if you know the filename beforehand, you can simply open the file in the editor pane and scroll through until you reach the line you want.

Once you’re there, simply click the line number and it will put a Line Breakpoint icon on that line so that you’d know where the breakpoint is set. You can remove the breakpoint by simply clicking on it again.

Now if you click the calculate age button, the program stops on the if condition. This part is exactly the same in both Chrome and Firefox.

Checking variable values

When you’re stopped at the if condition, if you want to see what is the value of a variable, simply have a look at the debugger pane’s Scope section in Chrome or Scopes section in Firefox.

Chrome

Firefox

Spotting the bug

Now let’s see where the bug is and how to fix it. If you have a look at the if condition more closely, you’ll see that there are two main parts in there. First one is m < 0 to check whether the month is less than zero, and the second is m = 0 && today.getDate() < birthDate.getDate() to check if the month is zero and today is less than birthday.

Now we know that someone’s used an equal sign instead of equality operator which has caused this bug. So let’s replace the = with === and then press Ctrl+S to save the changes. You can leave the breakpoint or remove it test with some dates to see if the bug is fixed or not. But after some tests we know that the bug is fixed. It’s time for you to go and write a test for this function so you won’t face the same bug again 😁.

Console window

Don’t forget that you can evaluate your expressions in the console window of the browser. As in you could simply put your condition in the console and see whether it’s returning true or false.

What else?

There are heaps of other breakpoints you could use such as conditional breakpoints which are triggered only if a condition you provide is true, breakpoints on caught and uncaught exceptions, and XHR breakpoints which are triggered when a URL matches a substring you’ve set. Let’s try a XHR breakpoint in , open up your DevTools, head over to debug pane and open the XHR Breakpoints section. Click in the box to add a line and enter raw and press Enter. Now refresh the page and you’ll see the breakpoint hit when the request is made. You can now use the same technique we saw earlier to step in the code and set a like breakpoint to debug further.

Small catch

One catch is that you need to make sure that the function you’re debugging is in the scope which is tricky if you’re using DevTools. In order to get it working you need to set a line breakpoint somewhere where the function is in scope, then trigger the breakpoint and call debug() in the DevTools console while the code is still paused on the line breakpoint.

Copy

(function () {
  function hello() {
    console.log('hello');
  }
  function world() {
    console.log(' world');
  }
  hello(yo); // This works.
  world();
})();
debug(hello); // This doesn't work. hey() is out of scope.

Summary

We saw how powerful the DevTools are and how much they can help us to find and fix bugs in our code. So stop using console.log and get to know your DevTools better. Happy coding from home ❤️.

What you need to know about Full screen API 💻

Tue, 10 Mar 2020 00:00:00 GMT

There are many situations where we’d like to see our web page in fullscreen mode. Be it games, online courses, video tutorials, or simply wanting more reading space while reading a book. What we usually do in these sort of scenarios is we manually set our browser’s tab/window in fullscreen mode (F11), but the result is not always what we expect, because some content is not designed to be viewed in fullscreen mode. Besides, what if we help our users to go fullscreen automatically instead, again it all comes to having a better user experience.

Intro

Full Screen API adds methods to the HTML Element to enable us to programmatically turn any content on a page to enter in fullscreen mode via JavaScript. WebGL, canvas, and video elements are the usual targets when it comes to going fullscreen. Online newspapers and magazines are other areas where this can be helpful.

So let’s see this in action first and then go through some details:

See the Pen Full Screen API Demo by Yaser Adel Mehraban (@yashints) on CodePen.

All we’re doing here is to call a method which is added by the Full Screen API:

Copy

const vid = document.getElementById('shuttle');

function toggleFullScreen() {
  if (!document.fullscreenElement && vid.requestFullscreen) {
    vid.requestFullscreen();
  } else {
    if (document.exitFullscreen) {
      document.exitFullscreen(); 
    }
  }
}

document.addEventListener("keydown", function(e) {
  if (e.keyCode == 13) {
    toggleFullScreen();
  }
}, false);

Exposed properties

There are two properties exposed on the document object:

💡 DocumentOrShadowRoot.fullscreenElement

The fullscreenElement property will give you the element that is displayed in fullscreen mode at the moment. It’s a way to check whether we’re in full screen mode or not.

💡 Document.fullscreenEnabled

The fullscreenEnabled property tells you whether or not the document is currently in a state that would allow fullscreen mode to be requested.

Exiting fullscreen

You’ll want to be sure to let your users know that they can press the Esc key (or F11) to exit fullscreen mode.

In addition, navigating to another page, changing tabs, or switching to another application (using, for example, Alt-Tab) while in fullscreen mode exits fullscreen mode as well.

As you saw in the example above, exiting the fullscreen mode can be done using document.exitFullscreen:

Copy

if (document.exitFullscreen) {
  document.exitFullscreen(); 
}

Feature detection

You can simply detect whether the feature is supported or not using either document.requestFullscreen or Element.requestFullscreen:

Copy

if(element.requestFullscreen) {
  element.requestFullscreen();
}

or:

Copy

if(document.requestFullscreen) {
  // enter fullscreen mode
}

Finding whether or not you’re in fullscreen mode

As mentioned earlier, the document.fullscreenElement is a good way to check whether the browser is in full screen mode or not. The value of this property will be null if we’re not in fullscreen, and the current fullscreen element otherwise.

⚠️ There is also a boolean property called fullscreen which is deprecated and should not be used. Although most of the browsers still support it.

Presentation

When toggling fullscreen mode programmatically, you’re responsible for how the elements will look like and what appears on the screen. It’s obvious you will need to change the width and height of the element to %100, and fortunately, there is a CSS pseudo-selector for you to use:

Copy

#shuttle:fullscreen {
  width: 100%;
  height: 100%;
}

Also you might want to hide some other elements on the screen to reduce clutter:

Copy

:fullscreen .hide {
  display: none;
}

Browser support

The browser support is really good at this point, almost all major browsers fully support it. Only IE11 and Safari have partial support. Opera mini and Android Browser do not support it at this point.

Hope you’ve learned a new trick if you didn’t know about this API and happy fullscreening (just made up the word 😁).

Stop gobbling up that memory 🛑

Fri, 28 Feb 2020 00:00:00 GMT

I’ve been writing web applications for more than a decade. From classic ASP to PHP, ASP.Net web forms and the list goes on and on. However, something that’s been common between all those years for me has been to care about how performing the site is. One important part of that has been to look out for memory leaks, because they can cause the page to go super slow or even crashing in more serious scenarios.

Intro

Memory leaks are a common problem in software development, regardless of whether the language you’re using is memory managed or not. By that I mean languages which come with a garbage collector. Memory leaks happen when a piece of memory is allocated, but never freed by the application, and is not returned to the container app or the operating system.

I remember going through the concept in uni, but I can’t remember anything apart from the fact that usually there is a tree made up of all the occupied memory locations. Every time the garbage collector is looking into the memory parses that tree, and if a node is not connected to any branch, it gets recollected and returned to the main program.

Most of us web developers are likely to use one of the major frameworks or libraries to write our applications. Some maybe using a bit older languages like PHP or Ruby, but no matter what we use, there will be a high chance that we come face to face with this problem one way or another.

Consequences

So what happens when there is a memory leak in our applications 🤔?

In some cases the memory consumption just keeps going up. If the user is using a decent spec machine, they might not even realise it. Not everyone is obsessed like us developers checking their task manager often to see how much memory is consumed.

Regardless, it slows down the page, makes interactions not responsive, and might even cause the tab or the whole window to crash.

Memory leak in JavaScript

It’s way easy in JavaScript to allocate some memory and forget about it. Even if you’re not writing plain JavaScript, it’s still possible that a memory leak happen, without you noticing it.

But how does it happen?

In JavaScript there are a few possible ways a memory leak can happen.

Unintentionally creating global variables
Timers and callbacks
Out of DOM references
Closures
Event listeners

Global variables

It’s very simple to create an unwanted global variable in JavaScript. Consider below code:

Copy

function helloWorld() {
  name = 'Yas';
  console.log(`Hello ${name}`);
}

In this simple function we have created a global variable called name. We didn’t want to, but we did it anyway.

💡 A reference to an undeclared variable creates a new variable inside the global object.

Same thing can happen if you use this:

Copy

function helloWorld(args) {
  this.name = 'Yas';
  console.log(`Hello ${name}`);
}

To prevent these sorts of leaks, use JavaScript in strict mode. You can do it by adding use strinct; at the top of your JavaScript file.

Even thought we consider accidental global variables one of the memory leak sources, there are still many global variables defined by the framework we use, or even the ones we intend to be there. Remember these are non collectable variables and unless nulled or reassigned, garbage collectors can’t do anything about them.

Timers and callbacks

The use of setInternal and setTimeout has gotten less and less popular as we’ve moved towards more modern concepts like async/await. Furthermore, some libraries and frameworks provide observables to facilitate callbacks. In which case they’re responsible for making sure the callbacks are unreachable once their own instance gets destroyed.

However, there are quite a number of scenarios where we need to use it in order to call a function at a later time or on a schedule.

Copy

let data = fetchData();
setInternal(function() {
  let node = document.querySelector('#list');
  // loop through data and create the html
  node.innerHTML = transform(data);
}, 1000)

This example shows how a timer can turn into something that cannot be collected. Even if the list node is removed from the DOM, the reference inside the handler remains active and cannot be collected. That causes their dependencies to be non collectable as well. So the data variable which might be really large, would sit in memory long after its need has ended.

Now let’s see how we can improve this to avoid a memory leak:

Copy

let node = document.querySelector('#list');
let data = fetchData();

function handler(data) {
  if(node) {
    // do stuff with data and create the list
    node.innerHTML = transform(data);
  }
};

setInterval(handler, 1000);

Out of DOM references (detached DOM)

When some nodes are removed from the DOM but still exists in memory through JavaScript, we have out of DOM references or detached DOM. Usually it means there is a reference to a variable which was referencing that node.

DOM is a doubly linked tree, meaning any reference to any node would mean the entire tree would not be garbage collected.

Let’s go through an example to make this a bit more clear:

Copy

function create() {
  let ul = document.createElement('ul');
  ul.id = 'list';
  for (var i = 0; i < 10; i++) {
    var li = document.createElement('li');
    li.textContent = `Item # ${i}`;
    ul.appendChild(li);
  }
  return ul;
}

const list = create();

document.body.appendChild(list);

function deleteList() {
  document.body.removeChild(document.getElementById('list'));
}

document.getElementById('delete').addEventListener('click', deleteList);

Clicking the delete button, will remove the list from the DOM, but there is a reference in JavaScript, so the list never gets garbage collected. We could identify detached node using heap snapshots in your browser DevTools. I am using Chrome here, but you can use Edge (similar to Chrome), and Firefox too.

And once the snapshot is taken, type detached in the filter text box and you’ll see the detached DOM nodes.

The way to fix these kind of issues is to always use local variables so that the reference is destroyed once the function execution is done.

Closures

Closures are one of the JavaScript’s features where most beginners struggle with. But once you’ve got it, it’s really easy to follow. In it’s core, a closure gives you access to an outer function’s scope from an inner function.

A more technical definition is that a closure is a combination of a function bundled together with references to it’s surrounding state (the lexical environment).

Copy

function init() {
  var hello = 'Hello'; // hello is a local variable created by init
  function helloWorld() { // helloWorld() is the inner function, a closure
    console.log(`${hello} world!`); // use variable declared in the parent function
  }
  helloWorld();
}
init();

Now that we know what’s a closure, let’s see how they can cause memory leaks. Imagine the following code:

Copy

var newElem;
 
function outer() {
    var someText = new Array(1000000);
    var elem = newElem;

    function inner() {
        if (elem) return someText;
    }

    return function () {};
}

setInterval(function () {
    newElem = outer();
}, 5);

In the above code, the inner function is never called, but it holds a reference to the elem. Keep in mind the scope of the inner is the same as function () {} which is returned by the outer function. Since newElem is a global variable, as long as a reference is pointing to function () {}, the shared context is preserved and someText is kept. Each call will then result in a remaining reference and as time goes by, we run out of memory eventually.

So what we can we do about these type of scenarios? First we need to stop using var. And furthermore, if we call the inner function like outer()(), then there won’t be any references left.

Event listeners

Every time you attach an event handler to a specific element, you need to keep a reference and remove it when you’re finished. So instead of:

Copy

function deleteList() {}

document.getElementById('delete').addEventListener('click', deleteList);

We should be doing:

Copy

function deleteList() {}

document.getElementById('delete').addEventListener('click', deleteList);

// do stuff

document.getElementById('delete').removeEventListener('click', deleteList);

Summary

We saw what will cause a potential memory leak in JavaScript and how to fix those issues, however, keep in mind that in most cases if you’re using a framework or library, these things are handled for you. If you’re using a library which you might suspect is causing memory leaks, you could easily find out using your browser DevTools memory profiler.

Hope this article has raised some awareness so that you can write more performing code and make user experience much better. No one wants to have their browser chew up memory like cheesecake right 😁?

Resources

More about Chrome DevTools memory profiler.
Find out about Firefox DevTools memory profiling.

Run code analysis with sonarqube using docker

Sun, 23 Feb 2020 00:00:00 GMT

I help some of my friends perform code reviews on their code bases from time to time as a side activity. I’ve used codelyzer before and it’s very similar to tslint in a sense. However, I wanted to test something new and thought let’s give SonarQube a shot this time.

Intro

SonarQube® is an automatic code review tool to detect bugs, vulnerabilities and code smells in your code. It can integrate with your existing workflow to enable continuous code inspection across your project branches and pull requests.

It has a server component where you can create projects and apply configurations like adding plugins and updating existing ones. And then there is a scanner component where it scans your code base and sends the result to the server to show in the SonarQube dashboard.

There are multiple versions of SonarQube but we’re going to use the community edition which is free and open source.

So without further ado, let’s get to it.

Setup

The documentation is really great, and there is a great two minute get started which is what I used to get my hand wet real quick. However, soon I realised that I have to install a lot of dependencies to get it working.

I was thinking how to avoid the extra installations, when suddenly a smile came to my face when I saw there is a docker image as well.

On the Docker Hub, you can find the information you need to get started, but for the purpose of this article, I just use a single command to get it going.

Copy

docker run -d --name sonarqube -p 9000:9000 sonarqube

This command will pull the image down and create a container from it. Run the container in the background mode and then exposes the port 9000 through to host.

Once done, you need to wait for a few mistunes for the server to get started. You can open a browser of your choice and go to localhost:9000 to see the server dashboard.

Use the default username as admin and the password admin to login.

Create a project

I don’t want to go deep into SonarQube here, but a project represents a code base where you can see the result of a scan done by the scanner. So once logged in, click on create a new project button in the centre of the page, then enter a unique project key and a display name.

On the next page enter a name for your project token. This is the key which scanner will use to authenticate to the server and send the result of a scan up.

Take note of this key as you can never see it once you’re navigated away from this page. Click continue and you’ll see a screen where you can specify your project configuration.

I wanted to run this on an Angular project with TypeScript, so I chose other, and then windows from the OS prompt.

Downloading the scanner

At this point you need to download the scanner and unzip it in a folder named sonarqube on your drive. I usually use c:\tools for these sort of usage (replace this with what you used if you chose to unzip it elsewhere).

Once done, open your scanner config file named sonar-scanner.properties from c:\tools\sonarqube\config folder and uncomment the line which specifies the server address.

Copy

#Configure here general information about the environment, such as SonarQube server connection details for example
#No information about specific project should appear here

#----- Default SonarQube server
sonar.host.url=http://localhost:9000

#----- Default source code encoding
sonar.sourceEncoding=UTF-8

Running a scan

Once done, you’re ready to go. Run the command below from your project root and wait for the scan to finish.

Copy

C:\tools\sonar\bin\sonar-scanner.bat -D"sonar.projectKey=fp" -D"sonar.sources=." -D"sonar.host.url=http://localhost:9000" -D"sonar.login=**********************************" -D"sonar.exclusions=node_modules/**/*"

Don’t forget to replace the key you got from the previous step in the sonar.login argument.

💡 In the last step when you downloaded the scanner zip file, the command you need to execute is written down for you ready to go, just add the exclude folder to avoid code analysis on external libraries.

It takes a while for the scan to finish based on the size of your code base. But once done, you can see the result in the SonarQube dashboard.

ℹ️ Remember to run npm install or yarn if you’ve just forked the code base from your git repository. I faced a few errors before I realised I didn’t have those installed and they are used by the scanner.

And once done, here is yow your dashboard will look like. Obviously the code base I was reviewing was a bit chaotic.

And you when you click on issues you can see them by category and also the rules used to perform the scan.

Summary

In terms of the speed, I was very happy with the result. It took nearly 2 minutes to scan my entire code base with more than 520 files and thousands line of code.

In term of the scan result, I would say based on the default rule set and considering that this was the community edition, the results are slightly better than tslint or codelyzer. I could easily navigate through issues, find out what line of code needed attention. It showed me the results with syntax highlighting and a lo of categories and tags where I could drill down and find what I wanted quickly.

So go ahead and give it a shot, I am sure you’ll find enough to keep you busy for a while to fix those. And bare in mind that it not only gives you linting errors, it will also give you some security hints based on OWASP top 10 as well.

Retro HTML, let's create marquees 😃

Sun, 09 Feb 2020 00:00:00 GMT

One of the first things I learnt to do with HTML was using marquee to move text across the screen horizontally. It was so cool to be able to create advertisements on a page and make users notice things with just a tag. Of course people overused it and then it became a out of fashion item in the new designs just like baggy jeans from 90s.

History

The HTML <marquee> element was used to insert a scrolling area of text. You could control what happened when the text reached the edge of it’s area using its attribute. Of course when I use past tense it doesn’t mean it’s not supported anymore. It became obsolete a while back, but most browsers still support it.

You can see it in action:

See the Pen Marquee by Yaser Adel Mehraban (@yashints) on CodePen.

CSS marquees

So I want to simulate some of these with just CSS. We’ll be using CSS animations and play with overflow to simulate text reaching to the edge.

Scrolling text

So let’s play with translateX() to specify the content placement at the start and end of the animation. We’ll use this to keep the text moving between start and end.

For the animation we will use an infinite linear with 10 seconds duration.

Copy

.marquee {
  height: 50px;
  overflow: hidden;
  position: relative;
}
.marquee span {
  font-size: 2em;
  color: turquoise;
  position: absolute;
  width: 100%;
  height: 100%;
  margin: 0;
  line-height: 50px;
  text-align: center;
  transform:translateX(100%);
  animation: cssmarquee 10s linear infinite;
}
@keyframes cssmarquee {
  0% {
  transform: translateX(100%);
  }
  100% {
  transform: translateX(-100%);
  }
}

And the HTML is simple:

Copy

<div class="marquee">
  <span>Scrolling Text </span>
</div>

See the Pen CSS-Marquee-ScrollText by Yaser Adel Mehraban (@yashints) on CodePen.

Slide in

Now let’s slide in the text and fix it at the beginning. We’ll use ease-out with a 200% value for translateX start and 0% for the end.

Copy

.marquee {
  height: 50px;
  overflow: hidden;
  position: relative;
}
.marquee span {
  font-size: 2em;
  color: green;
  position: absolute;
  width: 100%;
  height: 100%;
  margin: 0;
  line-height: 50px;
  text-align: left;
  animation: cssmarquee 10s ease-out;
}
@keyframes cssmarquee {
  0% {
    transform: translateX(200%);
  }
  100% {
    transform: translateX(0%);
  }
}

I won’t repeat the HTML code again to save some typing 😁. It’s exactly as above.

See the Pen CSS-Marquee-SlineIn by Yaser Adel Mehraban (@yashints) on CodePen.

Left to right

Now let’s move the text from left to right this time. Again, we’ll use a linear infinite animation with the reverse values for translateX as our first example.

The only other difference will be the initial value for transform property.

Copy

.marquee {
  height: 50px;
  overflow: hidden;
  position: relative;
}
.marquee span {
  font-size: 2em;
  position: absolute;
  width: 100%;
  height: 100%;
  color: green;
  margin: 0;
  line-height: 50px;
  text-align: center;
  /* Starting position */
  transform: translateX(-100%);
  animation: cssmarquee 20s linear infinite;
}
@keyframes cssmarquee {
  0% {
    transform: translateX(-100%);
  }
  100% {
    transform: translateX(100%);
  }
}

See the Pen CSS-Marquee-L2R by Yaser Adel Mehraban (@yashints) on CodePen.

Scroll vertically

By now you should be comfortable guessing what will be using. Instead of translateX, we will be using translateY.

Copy

.marquee {
  height: 100px;
  overflow: hidden;
  position: relative;
}
.marquee span {
  font-size: 2em;
  position: absolute;
  color: green;
  width: 100%;
  height: 100%;
  margin: 0;
  line-height: 50px;
  text-align: center;
  transform: translateY(-100%);
  animation: cssmarquee 5s linear infinite;
}
@keyframes cssmarquee {
  0% {
    transform: translateY(-100%);
  }
  100% {
    transform: translateY(100%);
  }
}

See the Pen CSS-Marquee-Down by Yaser Adel Mehraban (@yashints) on CodePen.

Bouncing text

Now let’s assume back then people wanted to get fancy. We want to bounce the text this time 😃. We will need to add alternate to our animation and reduce the delay. We will also add trnslateX so that the content doesn’t bounce off the whole page.

Copy

.marquee {
  height: 200px;
  overflow: hidden;
  position: relative;
}
.marquee span {
  font-size: 2em;
  position: absolute;
  color: turquoise;
  width: 100%;
  height: 100%;
  margin: 0;
  line-height: 50px;
  text-align: center;
  transform: translateY(70%);
  animation: cssmarquee 1s linear infinite alternate;
}
@keyframes cssmarquee {
  0% {
    transform: translateY(70%);
  }
  100% {
    transform: translateY(0%);
  }
}

See the Pen CSS-Marquee-Bounce by Yaser Adel Mehraban (@yashints) on CodePen.

Typewriting effect

Now this time we want to mix our animation with a bit of stepping to create a typewriter effect. We will be using from and to for our keyframes. Then use steps and step-end to simulate the text and blink cursor effect.

Copy

body {
  font-family: 'Special Elite', cursive;
  background: #efefef;
  padding-top: 5em;
  display: flex;
  justify-content: center;
}

.typewriter h1 {
  overflow: hidden; 
  border-right: .15em solid green; 
  white-space: nowrap;
  margin: 0 auto; 
  letter-spacing: .15em; 
  animation: 
    type 4s steps(50, end),
    blink-cursor .7s step-end infinite;
}

@keyframes type {
  from { width: 0 }
  to { width: 100% }
}

@keyframes blink-cursor {
  from, to { border-color: transparent }
  50% { border-color: green; }
}

Don’t forget to add the Special Elite font from Google fonts.

Copy

<head>
  <link href="https://fonts.googleapis.com/css?family=Special+Elite&display=swap" rel="stylesheet">
</head>

See the Pen CSS-Marquee-TypeWriter by Yaser Adel Mehraban (@yashints) on CodePen.

Summary

That was it. You can go back to whatever important job you were doing. A bit of distraction is always good for me and hope this bit was good for you too 😊. Until next time 👋🏼.

If you consider Puppeteer cool, then Playwright is awesomeness 😍

Fri, 24 Jan 2020 00:00:00 GMT

If you’ve ever used Puppeteer, you’d know it enables you to control a Chrome instance (or any other Chrome DevTools Protocol based browser) and execute common actions, much like in a real browser - programmatically, through a decent API. The same team has now build a new product called Playwright which apparently is their new favourite.

What is Playwright? 🤔

Playwright is a Node library which allows you to automate all major browsers - Chrome, Firefox, WebKit, and the new Microsoft Edge - plus the ability to execute actions, take screenshots, and much more, similar to Puppeteer.

In their own words:

Playwright is focused on enabling cross-browser web automation platform that is ever-green, capable, reliable and fast. Our primary goal with Playwright is to improve automated UI testing by eliminating flakiness, improving the speed of execution and offering insights into the browser operation.

Imagine the possibility of running end to end tests on all major browsers before you ship your any product to your users.

Is is replacing Puppeteer?

Based on what I see and how they’ve answered, I’d say YES. But with a promise to contain all of the functionality of Puppeteer, and make it a cross platform, vendor-neutral, and a collaboration effort between all major parties.

The lessons learnt are incorporated into it, APIs are testing-friendly, and it will be a cloud native product. For example all the user agent/device emulation is setup consistently on an object called BrowserContext to enable multi-page scenarios, click waits for the element to be available and visible by default, there is a way to wait for network and other events, etc.

Getting started

You can install Playwright via npm:

Copy

npm i playwright

This will install Playwright and its dependencies. Note that it will also download the browser binaries which are quite bulky (~50MB to ~150MB).

Launching a browser

Playwright module provides a method to launch a browser instance. The following is a typical example of using Playwright to drive automation. Below snippet shows opening a Chrome instance and browsing example.com page.

Copy

const playwright = require('playwright').chromium;  // Or 'firefox' or 'webkit'.

(async () => {
  const browser = await playwright.launch();
  const context = await browser.newContext();
  const page = await context.newPage('http://example.com');
  // other actions...
  await browser.close();
})();

Taking a screenshot of the page

You can execute most of the actions available on DevTools. For example we can take a screenshot of the page like so:

Copy

const pw = require('playwright');

(async () => {
  const browser = await pw.webkit.launch(); // or 'chromium', 'firefox'
  const context = await browser.newContext();
  const page = await context.newPage();

  await page.goto('https://www.example.com/');
  await page.screenshot({ path: 'example.png' });

  await browser.close();
})();

Emulate a device

Copy

const playwright = require('playwright').firefox;  
const iPhone = playwright.devices['iPhone 11 Pro'];

(async () => {
  const browser = await playwright.launch();
  const context = await browser.newContext({
    viewport: iPhone.viewport,
    userAgent: iPhone.userAgent
  });
  const page = await context.newPage('http://example.com');
  // other actions...
  await browser.close();
})();

Going beyond simple

Playwright allows you to easily emulate geo-location data.

Copy

const pw = require('playwright');
const pixel2 = pw.devices['Pixel 2'];

(async () => {
  const browser = await pw.chromium.launch();
  const context = await browser.newContext({
    viewport: pixel2.viewport,
    userAgent: pixel2.userAgent,
    geolocation: { longitude: 12.492507, latitude: 41.889938 },
    permissions: { 'https://www.google.com': ['geolocation'] }
  });

  const page = await context.newPage('https://maps.google.com');
  await page.click('text="Your location"');
  await page.waitForRequest(/.*pwa\/net.js.*/);
  await page.screenshot({ path: 'colosseum-android.png' });
  await browser.close();
})();

Note how we had to set the grant the necessary permission on the context, which means this is exactly what happens when a user browses the page.

Accessing elements on page

ElementHandle represents an in-page DOM element. ElementHandles can be created with the page.$ method, similar to what we’ve been using in DevTools.

Copy

const playwright = require('playwright').chromium;  // Or 'firefox' or 'webkit'.

(async () => {
  const browser = await playwright.launch();
  const context = await browser.newContext();
  const page = await context.newPage('https://example.com');
  const hrefElement = await page.$('a');
  await hrefElement.click();
  // ...
})();

Typing in text boxes

You can emulate a user typing into text fields, for example performing a search action:

Copy

const elementHandle = await page.$('input');
await elementHandle.type('Hello'); // Types instantly
await elementHandle.type('World', {delay: 100}); // Types slower, like a user
await elementHandle.press('Enter');

What else?

You can even play with mouse (pun intended):

Copy

// Using ‘page.mouse’ to trace a 100x100 square.
await page.mouse.move(0, 0);
await page.mouse.down();
await page.mouse.move(0, 100);
await page.mouse.move(100, 100);
await page.mouse.move(100, 0);
await page.mouse.move(0, 0);
await page.mouse.up();

What next?

The list of the available actions and what is available on the API documentation page. The list is almost overwhelming, in a good way 😁.

What browsers are supported?

Chromium: Playwright uses upstream versions of Chromium.
WebKit: Playwright makes a number of modifications to WebCore and WebKit2 in order to extend WebKit’s remote debugging capabilities and support the full set of Playwright APIs.
Firefox: Playwright makes a number of modifications to Firefox as well. Those are adding support for content script debugging, workers, CSP, emulation, network interception, etc. etc.
MS Edge: Since it’s based on Chromium, it’s supported as well.

What is the update cadence?

The team follow semver, and the team are trying to keep in sync with Chromium update cycle, possibly once a month.

But

Playwright is in a stage where there might be breaking changes coming up as it progresses through full support for those browsers. You can find more information around this on their Is Playwright Ready? page.

So go have fun and enjoy this awesome emerging web technology 👍🏼. I will be exploring this on a couple of my side projects too, so stay tuned for more posts.

Get to know the Clipboard API, be smarter with user interactions 📋

Thu, 09 Jan 2020 00:00:00 GMT

We face many situations in which we need to interact with user’s clipboard. Up until recently, browsers were using document.execCommand for clipboard interactions. It sounded great and it was (and still is) widely supported way to copy, cut, and paste into web apps, but the catch was that clipboard access is asynchronous and can just write to DOM.

Background

The fact that clipboard access is asynchronous means the user experience was suffering because a clipboard transfer was happening. This got worst when pasting because in most cases you need to sanitise the content can be safely processed.

It even got worst when permissions got into the equation, or the browser had to copy information from a link in the text and page had to wait for a network request, so the usage wasn’t that widespread.

Clipboard API

It was all sad and gloomy until the new Clipboard API was born. This API is a replacement for document.execCommand based copy and pasting that has a well defined permissions model and doesn’t block the page.

This API is a property of the window.navigator and is constructed from below properties and methods:

read()
readText()
write(data)
writeText(text)
addEventListener(event, handler)
removeEventListener(event, handler)
dispatchEvent(event)

To detect whether the feature is supported in the browser or not you can do:

Copy

if (navigator.clipboard) {
  // yep, happy coding.
} else {
  // oh no 😢. Use execCommand for now
}

In addition to that goodness, it can also read from and write to system clipboard 😍. Access to clipboard is behind Permissions API, which means without user’s permission none of these operations are permitted.

Copy text to clipboard

Text can be copied to the clipboard by calling the writeText method. This function returns a promise that will be resolved or rejected based on how the operation was performed.

Copy

navigator.clipboard.writeText('I am writing to clipboard 📋')
  .then(() => {
    console.log('Woohoo');
  })
  .catch(err => {    
    console.error('Oh no: ', err);
  });

On of the reasons the catch function might be called is that the permission is not granted by the user or they deny it.

You can use an async function and await:

Copy

async function writeTextToClipboard() {
  try {
    await navigator.clipboard.writeText('I am writing to clipboard 📋');
    console.log('Woohoo');  
  } catch (err) {
    console.error('Oh no: ', err);
  }
}

Read from clipboard

To read the copied text from clipboard, you can use the readText method and wait for the returned promise:

Copy

navigator.clipboard.readText()
  .then(text => {
    console.log('Pasted content: ', text);
  })
  .catch(err => {
    console.error('Oh no: ', err);
  });

And with async/await:

Copy

async function getCopiedTextFromClioboard() {
  try {
    const text = await navigator.clipboard.readText();
    console.log('Pasted content: ', text);
  } catch (err) {
    console.error('Oh no: ', err);
  }
}

Let’s see this in a demo.

See the Pen Clipboard API Demo by Yaser Adel Mehraban (@yashints) on CodePen.

Notice you had to give permission for it to work.

Handling paste

You can hook up to the (surprise) paste event on the document to be able to receive the text when it’s copied into clipboard and a paste action is triggered by user agent. If you want to modify the received data, you need to prevent the event from bubbling up.

Copy

document.addEventListener('paste', async (e) => {
  e.preventDefault(); // need to do something with the text
  try {
    let text = await navigator.clipboard.readText();
    text = text.toUpperCase();
    console.log('Pasted UPPERCASE text: ', text);
  } catch (err) {
    console.error('Oh no: ', err);
  }
});

Handling copy

Similar to paste, you can handle copy events too:

Copy

document.addEventListener('paste', async (e) => {
  e.preventDefault(); 
  try {
    for (const item of e.clipboardData.items) {
      await navigator.clipboard.write([
        new ClipboardItem({
          [blob.type]: blob
        })
      ]);
    }
    console.log('Image copied.');
  } catch (err) {
    console.error('Oh no: ', err);
  }
});

Permissions

The navigator.clipboard is only supported for pages served over HTTPS, plus clipboard access is restricted only when allowed to prevent further abuse. Active pages can write to clipboard, but reading requires granted permission.

There two new permissions which was introduced when Clipboard API was developed:

The clipboard-write is granted automatically to any active tab.
The clipboard-read must be requested.

Copy

const queryOpts = { name: 'clipboard-read' };
const permissionStatus = await navigator.permissions.query(queryOpts);
// Will be 'granted', 'denied' or 'prompt':
console.log(permissionStatus.state);

// Listen for changes to the permission state
permissionStatus.onchange = () => {
  console.log(permissionStatus.state);
};

Images

We don’t always use text, so the new write() method was introduced to allow us to write other data formats to the clipboard. This method is asynchronous too and truth is, writeText is calling this method behind the scenes.

Write to clipboard

To write an image to clipboard, you need to convert your image to a Blob. You can do it using a canvas element by calling the toBlob() function. At this point you can only pass one image at a time and multi image is under progress by Chrome team.

Copy

const canvas = document.getElementById("myCanvas");
const ctx = canvas.getContext('2d');

var img = new Image();
img.setAttribute('crossorigin', 'anonymous');
img.onload = function() {
  ctx.drawImage(img, 0, 0, 150, 150);
  img.style.display = 'none';

  canvas.toBlob(function(blob) {
    navigator.clipboard.write([
      new ClipboardItem({
        [blob.type]: blob
      })
    ]);
  })
};
  
img.src = 'https://image-url';

Read the image from clipboard

To read the image you’ll need to use the read() method, which reads the data and returns a list of ClipboardItem objects. You can use for...of which handles the async part for you as well.

Each returned object can have different type, so best option would be to call getType() on it and act accordingly. In our case the corresponding type would be Blob.

Copy

const items = await navigator.clipboard.read(); 

for (const clipboardItem of items) {
  for (const type of clipboardItem.types) {        
    if (type === "image/png") {
      console.log(type);
      const blob = await clipboardItem.getType(type);        
    }
  }
}

You can see a full demo here:

See the Pen Clipboard API Image Demo by Yaser Adel Mehraban (@yashints) on CodePen.

Browser support

The browser support for this API as of writing this article is not there yet. Chrome and Firefox are fully supporting both the Clipboard API and ClipboardEvent API.

Edge doesn’t support it and it’s been worked on in Safari.

Summary

We saw how easy and convenient it is to work with clipboard with this awesome API, and how much better the user experience will be.

So without further ado, happy coding 💻.

A year in review, what's coming 📆

Mon, 06 Jan 2020 00:00:00 GMT

I am back from a well deserved holiday and thought first thing first, I should review the last year and set some goals for the new one to be able to focus my energy to the right direction and learn from any mistake or missed opportunities.

A review on 2019

Last year was a super hectic year for me personally. I was working a full time consultancy job, advancing in my public speaking journey (more on this later), kicking some goals and missing others. All in all, it was a year with some ‘wow great’ and some ‘oh boy’ moments.

But let me start with the good parts first:

Learnt so many new topics and concepts.
Spoke at 8 major conferences and 7 meetups.
Got promoted to lead consultant.
Published 38 blog post.
Restarted going to gym two to three times a week.
Started on Dev.to and reached 15K followers, 170K post views, 6.5K reactions.
Published two npm packages.
Contributed to Open Source.
Read 60 books 🤯 (technical and non-technical).
Finished 10 side projects.
Met so many amazing people and made new friends.
And finished up with a position as a developer advocate for TelstraDev.

It was so good and I couldn’t believe what a little motivation could do.

However, this came with a cost, I felt nearly burnt down and exhausted from all of these activities, so I reduced them towards the end of the year and gave myself some time to recover. I didn’t touch Twitter (apart from posting some photos of my woodworking), didn’t check LinkedIn, read some books, restarted woodworking, enjoyed doing nothing basically.

I don’t want to go through too much detail, overall it was a great year.

2020, where to?

Starting 2020 for me was a with mixed feelings. I wanted to fix some errors I made last year, but at the same time I had to take some tough decisions. So here are my 2020 goals:

Be a better person.
Read more books.
Publish more targeted blog posts.
Take good care of my body and mind.
Learn more, listen more, talk less.
Continue public speaking, but be more selective, get more invitation, maybe a keynote/locknote, help others get started.
Be a caring and mindful advocate for developers out there.
Continue woodworking, spend more time with family, and spend less of weekends on technical subjects to balance lifestyle.
Be more productive by sleeping more, eating healthy food, and more exercise.
Make new friends, surround myself with good people.
And a nice to have one is to start a video podcast on web and front end.

This list is not a far fetched list, it’s a carefully drafted one based on last year’s review and feedback.

And I am so grateful for the support I’ve received by all of you. Keep your positive energy coming and stay tuned for some quality content coming up.

Image aspect ratio done right 😍

Thu, 19 Dec 2019 00:00:00 GMT

If you have an image on the page which is 600x200 pixels, what is the chance of it shown exactly with that size? I’d say not likely, because most of the time it’s put inside a container which defined the width and the image is set to width: %100. But the size of the image is not that important at this point, what is important is that you will have a DOM reflow once the image is loaded.

Problem

Let’s see this in action. I’ll use Glitch to quickly whip up a demo. We need three divs, two of those contain a piece of text and the one in between contains an image.

Copy

<head>
  <title>Image aspect ratio</title>
  <style>
    .image-container {
      width: 500px;
    }

    img {
      width: 100%;
    }
  </style>
</head>
<body>
  <h1>Image aspect ratio done right</h1>
    
  <div >
    <p>
      This is the first bit of text here!
    </p>        
  </div>
  
  <div class="image-container">
    <img src="https://www.hindustantimes.com/rf/image_size_960x540/HT/p2/2018/09/08/Pictures/_7ae928c8-b34e-11e8-bb15-a1f88311a832.jpg"
          width="960" height="540"/>
  </div>
  
  <div>
    <p>
      This is the second bit of text here!
    </p>  
  </div>
</body>

You can see the full page in this Glitch app.

If we load the site under a slow network now (on high speed network you can’t see much of a difference), you will the page load like this:

First the two paragraphs appear close to each other, then when the network request is finished, the image fills its gap and there is a DOM reflow performed to push the second paragraph down. How sad is that 😥?

Solution #1

The very first thing we can do is to simply set the image height to auto. This is a new feature which has landed in FireFox and Chrome. This will tell the browser to find out the metadata from image attributes, calculate the necessary height and reserve that space. So all we need to do is this:

Copy

<style>
  img {
    width: 100%;
  }
</style>

And we we reload the page on the same network, you can see the page load like this:

How good is that 😍?

Solution #2

There is a new proposal in a draft state which is introducing a new CSS property called, well aspect-ratio. With this you can simply add the property to img tag and use the width and height attributes to tell the browser how much space should be reserved.

Copy

img {
  aspect-ratio: attr(width) / attr(height);
}

This has the benefit of being super simple, readable, and supporting most of the images on the web because many people already have put width and height attributes on their images.

On the other hand, a junior developer who’s starting with HTML and CSS doesn’t have any trouble understanding what’s happening and how little should be done to do something properly.

This has been implemented in Firefox, Chrome and Edge.

Summary

Getting image aspect ratio is very important especially if it impacts your site’s user experience. So knowing how to fix that issue is also important. Hope you’ve enjoyed this article and till next time 👋🏼.

Tensorflow.js available on WebAssembly backend 🔥

Tue, 17 Dec 2019 00:00:00 GMT

Tensorflow.js is a library which lets you perform machine learning in the browser or in Node. It uses the GPU or CPU to do training and calculation, but recently the team have done a great job and brought WebAssembly backend to its ecosystem so that you can perform predictions faster. So without further ado, let’s deep dive into this greatness.

Scenario

Let’s assume you want to perform object detection in a given image. For this you can use multiple models, but for now let’s focus on MobileNet.

We will use parcel to setup our app.

Setup without WASM backend

Packages

We will need to have a package.json file with below setup:

Copy

{
  "name": "wasm-parcel",
  "version": "1.0.0",
  "description": "Sample parcel app that uses the WASM backend",
  "scripts": {
    "watch": "parcel index.html --open",
    "build": "parcel build index.html"
  },
  "dependencies": {
    "@tensorflow/tfjs": "^1.4.0",
    "@tensorflow/tfjs-backend-wasm": "1.4.0-alpha3"
  },
  "browserslist": [
    "defaults"
  ],
  "devDependencies": {
    "@babel/core": "7.7.5",
    "@babel/plugin-transform-runtime": "^7.7.6",
    "@babel/preset-env": "^7.7.6",
    "parcel-bundler": "^1.12.4",
    "parcel-plugin-static-files-copy": "^2.2.1"
  },
  "keywords": []
}

HTML

Let’s setup our app like we do normally with Tensorflow.js. We need to add a div to show the status, and another div which contains an image tag:

Copy

<div class="tfjs-example-container">
  <section class="title-area">
    <h1>TensorFlow.js running on WebAssembly backend</h1>
  </section>

  <section>
    <p class="section-head">Status</p>
    <div id="status"></div>
  </section>

  <section>
    <p class="section-head">Image used</p>

    <img id="img" src="https://cdn.glitch.com/6e79f1d1-4e57-42bc-a2aa-0aaca87b6f6d%2Fpiano.jpg?v=1576551575229" width="224px" />
  </section>
</div>

And let’s our JavaScript file before body close tag:

Copy

<script src="index.js"></script>

Altogether your HTML should look something like this:

Copy

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Hello!</title>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <link rel="stylesheet" href="style.css" />
  </head>
  <body>
    <div class="tfjs-example-container">
      <section class="title-area">
        <h1>TensorFlow.js running on WebAssembly backend</h1>
      </section>
  
      <section>
        <p class="section-head">Status</p>
        <div id="status"></div>
      </section>
  
      <section>
        <p class="section-head">Image used</p>
  
        <img id="img" src="https://cdn.glitch.com/6e79f1d1-4e57-42bc-a2aa-0aaca87b6f6d%2Fpiano.jpg?v=1576551575229" width="224px" />
      </section>
  
      <script src="index.js"></script>
    </div>
  </body>
</html>

CSS

Let’s add some basic styling so that it’s not an ugly app when we run it. We’re doing some cool stuff here and it deserves good look 😉:

Copy

/* CSS files add styling rules to your content */

body {
  font-family: helvetica, arial, sans-serif;
  margin: 2em;
}

p {
  max-width: 960px;
  line-height: 1.6em;
}

p.section-head {
  font-variant: small-caps;
  text-transform: uppercase;
  letter-spacing: 0.17em;
  line-height: 1.2em;
  font-weight: 500;
  margin-top: 2em;
  margin-bottom: 1em;
  border-left: 2px solid #ef6c00;
  padding-left: 24px;
  margin-left: -24px;
  color: #818181;
}

JavaScript

In the code we need to perform two main operations, load the model, and use it for prediction. So lets load the model first:

Copy

import * as tf from "@tensorflow/tfjs";

let model = await tf.loadGraphModel(
  "https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/classification/2",
  { fromTFHub: true }
);

And we get a reference to the image tag, use browser.fromPixel method of Tensorflow.js to load the image and normalise it. And at last, feed it into predict method of our model:

Copy

const imgElement = document.getElementById("img");

let img = tf.browser
    .fromPixels(imgElement)
    .resizeBilinear([224, 224])
    .expandDims(0)
    .toFloat();

const prediction = model.predict(img);

Since we want to compare the timing between the two approach, let’s add some timers into our method and measure how long the operation would take. Altogether your JavaScript file should look like this:

Copy

import * as tf from "@tensorflow/tfjs";

function status(text) {
  document.getElementById("status").textContent = text;
}

async function main() {
  
  let model = await tf.loadGraphModel(
    "https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/classification/2",
    { fromTFHub: true }
  );

  const startTime1 = performance.now();  

  const imgElement = document.getElementById("img");
  status("Model loaded!");

  let img = tf.browser
    .fromPixels(imgElement)
    .resizeBilinear([224, 224])
    .expandDims(0)
    .toFloat();
  
  let startTime2 = performance.now();
  
  const logits = model.predict(img);

  const totalTime1 = performance.now() - startTime1;
  const totalTime2 = performance.now() - startTime2;
  status(`Done in ${Math.floor(totalTime1)} ms ` +
      `(not including preprocessing: ${Math.floor(totalTime2)} ms)`);

  const values = await logits.data();
  console.log(values);  
}

document.addEventListener("DOMContentLoaded", main);

.babelrc

If you don’t setup the babel plugins properly, you will get an error like below:

❌ Uncaught ReferenceError: regeneratorRuntime is not defined at HTMLDocument.main (index.js:8)

That’s because we’re using an async function on the top level. Add the below config to your .babelrc file to get around the error.

Copy

{
  "presets": ["@babel/preset-env"],
  "plugins": ["@babel/plugin-transform-runtime"]
}

Running the app

Now you can run yarn followed by yarn watch and a browser window opens with the app inside. You should see a page like this:

Note the time taken to predict what’s in the picture. Now lets add the WebAssebly backend and run the app to see how it performs.

With WASM backend

In order to add the WebAssembly backend you need to install the @tensorflow/tfjs-backend-wasm package:

Copy

{
  ...,
  "dependencies": {
    "@tensorflow/tfjs": "^1.4.0",
    "@tensorflow/tfjs-backend-wasm": "1.4.0-alpha3"
  },
  ...
}

Next step is to let Parcel know how to load the WASM file. When the WASM backend is initialized, there will be a fetch / readFile for a file named tfjs-backend-wasm.wasm relative to the main JS file. That’s why we need to use this section if we’re using a bundler.

Copy

{
  ...,
  "staticFiles": {
    "staticPath": "./node_modules/@tensorflow/tfjs-backend-wasm/dist",
    "excludeGlob": [
      "**/!(*.wasm)"
    ]
  },
  ...
}

And the last thing we need to do is to tell Tensorflow.js to use this backend:

Copy

import "@tensorflow/tfjs-backend-wasm";

async function main() {  
  await tf.setBackend("wasm");
  //...
}

And that’s all you need in order to enable the WebAssembly backend. Now let’s run the app and see the difference:

Copy

yarn && yarn watch

You should see the app compiled and a browser window open:

And boom 🤯, look at the difference there. Almost down by 2 seconds, and that’s just doing a single operation. Imagine if we were doing more operations and the benefits we gain from this approach.

Summary

You can find the full demo on my GitHub repo.

This feature is defo one of the best thing that’s happened since introducing Tensorflow.js to allow web developers get into ML and AI within the browser in my opinion. So go ahead use this feature and benefit from the massive performance improvements.

However, there is a catch 👇🏼:

What’s the catch

The catch is that not all functions are implemented in the WASM backend. It means you can’t run many of the demo’s on it unless the team implement them. You can follow the progress on their GitHub repo to stay on top of the game.

Transfer learning with Tensorflow.js

Mon, 16 Dec 2019 00:00:00 GMT

Tensorflow.js is a library which lets you perform machine learning in the browser or in Node. A while ago I wrote an intro on it and also gave a few talks including one which had a live demo where I re-trained the Speech Command model and flew a drone using my voice 😍. Many people asked about the details, so I decided to blog the whole thing down.

Intro

Speech Command is a machine learning model which allow you to classify 1-second audio snippets from the speech command dataset which has about 18 basic words (i.e. up, left, right, down, yes, no, etc) in it at the moment.

In this article we want to expand the dataset using three words which we will use to fly a drone with. To do this we will be using a method called transfer learning, where we will use the pre-trained model and retrain it to learn new words.

Requirements

You will need some basic knowledge around web development and JavaScript, and nothing else.

Get started

We will be using Glitch to quickly implement the app. So go ahead andremix this starter template.

First thing first we should add the tensorflowjs and speech-command script to our page:

Copy

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Transfer learning with Tensorflow.js and Speech Command</title>
    <script src="https://unpkg.com/@tensorflow/tfjs"></script>
    <script src="https://unpkg.com/@tensorflow-models/speech-commands"></script>
  </head>
  <body>
    <h1>Tensorflow.js</h1>
    <div id="info"></div>
    
    <!-- include the Glitch button to show what the webpage is about and
          to make it easier for folks to view source and remix -->
    <div class="glitchButton" style="position:fixed;top:20px;right:20px;"></div>
    <script src="https://button.glitch.me/button.js"></script>
    
    <!-- import the webpage's javascript file -->
    <script src="/script.js" defer></script>
  </body>
</html>

You can leave the default Glitch button for now or delete it if you want. That just gives you ability to share it with others after it’s finished so they can remix and use it.

We will use the <div id="info"> tag to show output of model and training on the screen.

Load the model

Now we need to load the model which is very simple. All you need to do is to call the create method on the speech commands’ object. The BROWSER_FFT argument passed to it is just telling speech command to use the browser’s native Fourier transform. For more information regarding Speech Command’s APIs have a look at their ReadMe.

Copy

let recogniser;

async function app() {
 recogniser = speechCommands.create('BROWSER_FFT');
 await recogniser.ensureModelLoaded();
}

app();

💡 Note: If you’re using your local editor and npm to pull down the packages, you need to import the speechCommand:

Copy

import * as speechCommands from '@tensorflow-models/speech-commands';

Collecting data

It’s time to collect our sample audio to be able to feed it into our model. Our intention is to add three words (take off, land, and flip) to Speech Command dataset. For this we use three buttons to represent our words (labels in the world of machine learning) and another to capture background noise. The background noise bit is really important because it can make or break the whole setup (my first demo failed just because of this).

Copy

<button id="takeoff" onmousedown="collectSample(0)" onmouseup="collectSample(null)">Take Off</button>
<button id="Land" onmousedown="collectSample(1)" onmouseup="collectSample(null)">Land</button>
<button id="flip" onmousedown="collectSample(2)" onmouseup="collectSample(null)">Flip</button>
<button id="noise" onmousedown="collectSample(3)" onmouseup="collectSample(null)">Noise</button>

And we will need to implement the collectSample method:

Copy

const NUM_FRAMES = 6;
let samples = [];

function collectSample(label) {
 if (recogniser.isListening()) {
   return recogniser.stopListening();
 }
 if (label == null) {
   return;
 }
 recogniser.listen(async ({spectrogram: {frameSize, data}}) => {
   let vals = normalise(data.subarray(-frameSize * NUM_FRAMES));
   samples.push({vals, label});
   document.querySelector('#info').textContent =
       `${samples.length} sample collected`;
 }, {
   overlapFactor: 0.999,
   includeSpectrogram: true,
   invokeCallbackOnNoiseAndUnknown: true
 });
}

function normalise(x) {
 const mean = -100;
 const std = 10;
 return x.map(x => (x - mean) / std);
}

Let’s go through what’s happening here. The collectSample method associates a label with the output of recogniser.listen(). The includeSpectrogram flag makes sure that recogniser gives the raw data frequency of 1 sec of audio which then is divided into 43 frames.

Because we’re aiming for short sound tracks, we will take into accounts only the last 6 frames (~140ms), enough to give us time to pronounce the words:

Copy

let vals = normalize(data.subarray(-frameSize * NUM_FRAMES));

And last, to avoid any numerical issues, we normalise the data to have an average of 0 and a standard deviation of 1. In our case the spectrogram values are usually large negative numbers around -100 with deviation of 10.

All this info is saved to the samples variable.

Feel free to test your sample collection to see if everything works fine so far.

Train the model

Now we need to build a model with the correct architecture (like Speech Command) and feed our sample data to it for training. We will use another button to trigger the training:

Copy

<br/><br/>
<button id="train" onclick="train()">Train</button>

Copy

const INPUT_SHAPE = [NUM_FRAMES, 232, 1];
let model;

async function train() {
 toggleButtons(false);
 const ys = tf.oneHot(samples.map(e => e.label), 3);
 const xsShape = [samples.length, ...INPUT_SHAPE];
 const xs = tf.tensor(flatten(samples.map(e => e.vals)), xsShape);

 await model.fit(xs, ys, {
   batchSize: 16,
   epochs: 10,
   callbacks: {
     onEpochEnd: (epoch, logs) => {
       document.querySelector('#info').textContent =
           `Accuracy: ${(logs.acc * 100).toFixed(1)}% Epoch: ${epoch + 1}`;
     }
   }
 });
 tf.dispose([xs, ys]);
 toggleButtons(true);
}

function buildModel() {
 model = tf.sequential();
 model.add(tf.layers.depthwiseConv2d({
   depthMultiplier: 8,
   kernelSize: [NUM_FRAMES, 3],
   activation: 'relu',
   inputShape: INPUT_SHAPE
 }));
 model.add(tf.layers.maxPooling2d({poolSize: [1, 2], strides: [2, 2]}));
 model.add(tf.layers.flatten());
 model.add(tf.layers.dense({units: 3, activation: 'softmax'}));
 const optimizer = tf.train.adam(0.01);
 model.compile({
   optimizer,
   loss: 'categoricalCrossentropy',
   metrics: ['accuracy']
 });
}

function toggleButtons(enable) {
 document.querySelectorAll('button').forEach(b => b.disabled = !enable);
}

function flatten(tensors) {
 const size = tensors[0].length;
 const result = new Float32Array(tensors.length * size);
 tensors.forEach((arr, i) => result.set(arr, i * size));
 return result;
}

And we add a call to build model in our entry function when the app loads:

Copy

async function app() {
  recogniser = speechCommands.create("BROWSER_FFT");
  await recogniser.ensureModelLoaded();

  document.getElementById("info").textContent = "Model loaded";

  buildModel();
}

At this point, you can see a train button on the page and if you click it after you’ve collected some sample data, you can see the training happen and the accuracy of the model shown after it’s finished.

So let’s break this down because it does look a bit busy. First thing first we need to build a model with the right architecture. This model will have four layers, a convolutional layer that processes the audio data, a max pool layer, a flatten layer, and a dense layer that maps to all three actions.

The input shape of the model is [NUM_FRAMES, 232, 1] where each frame is 23ms of audio containing 232 numbers that correspond to different frequencies. In this demo we’re using samples that are 6 frames long (~140ms) because we’re using short words to control the drone.

For the training part, we’re using 1- epochs (imagine the number of iterations which model uses to go through running it’s operations end to end).

Capturing the words

It’s time to capture a word, feed it into the model and see how accurate it is. Let’s add another button to trigger listening:

Copy

<br/><br/>
<button id="listen" onclick="listen()">Listen</button>

Then we need to listen to the microphone through browser and feed that into the model to see what is the output.

Copy

async function controlDrone(labelTensor) {
  const label = (await labelTensor.data())[0];
  if (label == 3) {
    return;
  }
  let recognisedWord = "none";
  switch (label) {
    case 0:
      recognisedWord = "take off";
      break;
    case 1:
      recognisedWord = "land";
      break;
    case 2:
      recognisedWord = "flip";
      break;    
  }
  document.getElementById("info").textContent = recognisedWord;
}

function listen() {
  if (recogniser.isListening()) {
    recogniser.stopListening();
    toggleButtons(true);
    document.getElementById("listen").textContent = "Listen";
    return;
  }
  toggleButtons(false);
  document.getElementById("listen").textContent = "Stop";
  document.getElementById("listen").disabled = false;

  recogniser.listen(
    async ({ spectrogram: { frameSize, data } }) => {
      const vals = normalise(data.subarray(-frameSize * NUM_FRAMES));
      const input = tf.tensor(vals, [1, ...INPUT_SHAPE]);
      const probs = model.predict(input);
      const predLabel = probs.argMax(1);
      await controlDrone(predLabel);
      tf.dispose([input, probs, predLabel]);
    },
    {
      overlapFactor: 0.999,
      includeSpectrogram: true,
      invokeCallbackOnNoiseAndUnknown: true
    }
  );
}

Again, let’s break this down a bit. The listen method will listens to the microphone real time and feeds that into the model for predictions. The logic is very similar to collectSample method where we normalise the data and drop all frames except the NUM_FRAMES (6) frames. The difference is that we call model.predict method to see what is been spoken.

The output of the prediction is a tensor with the shape of [1, numClasses] representing a probability distribution over all our classes (labels). We will use the first one returned to as it will be the strongest match using probs.argMax(1).

Now we can call the .data() and get it’s first argument to get our label from it:

Copy

const label = (await labelTensor.data())[0];

I’ve used a switch case for simplicity, but feel free to chuck them all in a set and use it’s index.

Note: It’s very important to dispose the output tensors to clean up the memory through a call to tf.dispose([input, probs, predLabel]);.

Test the whole demo

It’s time to give our full demo a shot. Record the samples and the background noise, then train the model and then verify it using the listen button.

💡 For my demo, I had to use a recording of the drone’s flying in order to be able to run it successfully. That’s because the first command always works because there is no background noise, but once the drone is up, the noise of the drone will mask the consequent words 😁. So when recording noise, use a drone’s actual noise which you can find on YouTube.

See the full working demo on Glitch.

I will do another post on how to wire this up with the server side code to fly the drone 🚁 (think of a drone when you look at the helicopter emoji 😁).

State management in Vue.js

Mon, 18 Nov 2019 00:00:00 GMT

Vue.js is an approachable, versatile, performant, and progressive framework to build user interfaces with. I’ve written three separate articles on it so far:

We will be looking at state management in Vue.js this time which is the centre of communication in large applications.

Why do we need state management

When dealing with large applications, we definitely have to pass information between components. Shopping card needs to know whether the user is logged in or not, the search needs to get updated based on changes to filter data, and many more scenarios.

The usual ways of passing data such as props only gets us just that far. Things get complex when we want to pass the data in multiple layers of component hierarchy. Or even when we want to pass data into components outside of those hierarchies.

At the end of the day, it’s important to have a single source of truth and that’s what Vue is offering to solve using vuex. But consider using a state management library if:

Multiple views may depend on the same piece of state.
Actions from different views may need to mutate the same piece of state.

Vuex

vuex is an Elm-inspired state management library. It integrates really well into vue-devtools, providing zero-setup access to time travel debugging.

Vuex is based on Flux (a state management design pattern introduced by Facebook). The TLDR version looks like:

Installation

If you haven’t used Vue CLI to create your project, there are three steps you need to take:

Install the library:

Copy

npm install vuex --save

Import and wire it up:

Copy

import Vue from 'vue'
import Vuex from 'vuex'

Vue.use(Vuex)

And last, because Vuex requires Promise you should include a package like es6-promise and import it before using Vuex:

Copy

npm install es6-promise --save

And:

Copy

import 'es6-promise/auto'

💡 This library will automatically use a polyfill if the browser doesn’t support ES6’s Promise.

Getting started

In its simplest form, you can use a centralised store as such:

Copy

const store = new Vuex.Store({
  state: {
    count: 0
  },
  mutations: {
    increment (state) {
      state.count++;
    }
  }
})

Now, you can access the state object as store.state, and trigger a state change with the store.commit function:

Copy

console.log(store.state.count) // -> 0

store.commit('increment')

console.log(store.state.count) // -> 1

Vuex state

This library uses a single state tree which ensures that the source of truth is always the state. However, this doesn’t have any conflicts with the modularity. We will have a look at the sub-modules later.

Vuex’s state consist of four objects, state, mutations, actions, and getters. So lets see what each of these are. But first, let’s see how we can use the store in our components. We use our previous students example from here on.

Including store in your components

Vuex provides a mechanism to inject the store into all child components from the root component with the store option which happens when you use Vue.use(Vuex):

Copy

import Vue from 'vue'
import Vuex from 'vuex'

Vue.use(Vuex)

const store = new Vuex.Store({
  state: {
    students: [
      {
        name: 'Yas',
        age: 25
      }
    ]
  }
})

const app = new Vue({
  el: '#app',  
  store
})

This will allow you to access state using this.$store in any child component.

Copy

const Students = {
  template: `
    <ul>
      <li v-for="student in students">{{ student.name }}</li>
    </ul>`,
  computed: {
    students () {
      return this.$store.state.students
    }
  }
}

Note how I accessed the student in a computed property. This will ensure that we rerender the page whenever store changes.

How to use multiple properties from state

If you have to use multiple properties from store, Vuex offers the mapState helper method. Using this method you have the ability to map multiple properties from state to your computed properties:

Copy

import { mapState } from 'vuex';

export default {
  name: "Students",
  computed: mapState({
    students: state => state.students,
    teachers: state => state.teachers
  })
};

As you can see, you can use an arrow function to retrieve the state and access it’s properties. You can also define aliases to keep your code cleaner and avoid repeating state.property everywhere you want to use it.

If the local and store property names matches, you can skip the arrow function and just pass an array of property names you want to have:

Copy

import { mapState } from 'vuex';

export default {
  name: "Students",
  computed: mapState([
    'students',
    'teachers'
  ])
};

If you have some local computed properties, you could use the spear operator to mix them with mapState:

Copy

import { mapState } from 'vuex'

export default {
  // ...
  computed: {
    myLocalProp() {
      return {
        name: 'Yas'
      }
    },

    ...mapState({    
      numOfStudents: state => state.students.length,
      students: state => state.students
    })
  }
}

Getters

Sometimes you need to have derived properties based on store’s state. For example if you want to know how many students are older than 25. You can implement this in your component like so:

Copy

<template>
  <div>Total number of students aged 25 or older is {{numOfStudents}}</div>
</template>
<script>
import { mapState } from 'vuex'

export default {
  // ...
  computed: mapState({    
    numOfStudents: state => state.students
      .filter(s => s.age > 24).length;
  })
}
</script>

But this means that if another component wants to have this logic, they need to duplicate the code. A better approach to these sorts of derived state is to use getters.

Copy

const store = new Vuex.Store({
  state: {
    students: [
      { id: 1, name: 'Yas', age: 26 }
    ]
  },
  getters: {
    numOf25OrOlderStudents: state => {
      return state.students
        .filter(s => s.age > 24).length;
    }
  }
})

These sort of properties appear under store.getters:

Copy

computed: {
  numOf25OrOlderStudents () {
    return this.$store.getters.numOf25OrOlderStudents
  }
}

Similar to mapState, you have mapGetter so you can get these properties easier:

Copy

computed: {
  
  ...mapGetters([
    'numOf25OrOlderStudents',
  ])
}

Mutations

A mutation is the only way you can change the state in Vuex. These mutations are basically events which have two properties:

A string type
A handler

You’ll perform the modifications inside the handler and it will receive the state as it’s first argument:

Copy

const store = new Vuex.Store({
  state: {
    students: [
      { id: 1, name: 'Yas', age: 26 }
    ]
  },
  mutations: {
    addStudent (state, student) {
      // mutate state
      state.students.push(student)
    }
  }
})

💡 You can’t call a mutation handler directly, instead, you will need to call store.commit method.

Copy

const student = { id: 2, name: 'Alex', age: 23 }

store.commit('addStudent', student)

Like React Redux’s actions, it’s best to use constants for these mutation types:

Copy

// mutation-types.js
export const ADD_STUDENT_MUTATION = 'ADD_STUDENT'

Copy

// store.js
import Vuex from 'vuex'
import { ADD_STUDENT_MUTATION } from './mutation-types'

const store = new Vuex.Store({
  state: { ... },
  mutations: {
    [ADD_STUDENT_MUTATION] (state, student) {
      state.students.push(student)
    }
  }
})

The mutations should be synchronous. This is an important rule which you should adhere to so that Vuex can capture before and after mutation states. We will see how to have async state mutations using actions.

Actions

Actions are very similar to mutations, but there are two differences between the two. The first one is that instead of mutating the state themselves, they will commit mutations. And the second one is that actions can have asynchronous operations.

Copy

const store = new Vuex.Store({
  state: {
    students: [
      { id: 1, name: 'Yas', age: 26 }
    ]
  },
  mutations: {
    addStudent (state, student) {
      // mutate state
      state.students.push(student)
    }
  },
  actions: {
    addStudent (context, id) {
      const student = await apiService.get(id)

      context.commit('addStudent', student)
    }
  }
})

The first thing to note here is that actions won’t receive the store object, they will get a context object which exposes the same properties. We will see why when we review the modules. And the second one is that there is an asynchronous operation happening here.

You can use ES6 Object Destructuring to get only commit passed down to the action. This is useful if you have multiple commits in your actions because it will keep the code way less.

Copy

actions: {
    async addStudent ({ commit }, id) {
      const student = await apiService.get(id)

      commit('addStudent', student)
    }
  }

You can trigger an actions using store.dispatch method.

Copy

store.dispatch('addStudents', 2)

Again, similar to mapState you have mapActions to be able to dispatch an action in your components instead of using this.$store.dispatch method which is way much cleaner. The difference is, you will call this in your method objects:

Copy

import { mapActions } from 'vuex'

export default {
  // ...
  methods: {
    ...mapActions([
      'addStudent', // map `this.addStudent()` to `this.$store.dispatch('addStudent')`

      // `mapActions` also supports payloads:
      'addStudent' // map `this.addStudent(id)` to `this.$store.dispatch('addStudent', id)`
    ]),
    ...mapActions({
      add: 'addStudent' // map `this.add()` to `this.$store.dispatch('addStudent')`
    })
  }
}

Modules

As we’ve seen so far, the state can easily get super big as the project grows. This can lead to multiple issues, but we can use modules to break this single state into multiple modules which have their own state, getters, mutations, actions, and guess what, sub modules 😍.

Copy

const studentStore = {
  state: { ... },
  mutations: { ... },
  actions: { ... },
  getters: { ... }
}

const teachersStore = {
  state: { ... },
  mutations: { ... },
  actions: { ... }
}

const store = new Vuex.Store({
  modules: {
    a: studentStore,
    b: teachersStore
  }
})

store.state.studentStore // -> `studentStore`'s state
store.state.teachersStore // -> `teachersStore`'s state

Inside the state and getters, the module’s state is passed down.

Copy

const studentStore = {
  state: { students: [] },
  mutations: {
    addStudent (state, student) {
      // `state` is the local module state
      state.students.push(student)
    }
  },

  getters: {
    count (state) {
      return state.students.length
    }
  }
}

However, inside the actions, you can access the global store from context via context.rootStore:

Copy

const moduleA = {
  // ...
  actions: {
    async addStudentIfRegistrationOpen ({ commit, rootState }, id) {
      if (rootState.registrationOpen) {
        const student = await apiService.get(id)

        commit('addStudent', student)
      }
    }
  }
}

💡 By default the getters, mutations, and actions are registered on the global namespace. This will allow multiple modules to react to same mutation or action if they want to. You can use the namespaced: true flag on the module to make it self contained.

You can find out more about the modules in the official documentation.

Summary

Alrighty, you now know enough about state management in Vue using Vuex to start using it. But beware that there are many other little tips and tricks where I thought would fall outside of a single article which you should look into. For now enjoy playing with this amazing library and get your hands wet.

PS: if you want a head start, use this GitHub repository and create a code sandbox out of it. It contains basic setup and modules as well.

Network Information API, another step towards a better web 😍

Mon, 18 Nov 2019 00:00:00 GMT

This year has been full of articles around how the web is bloated with static assets and how this has impacted users to face a poor user experience when using not so good networks around the world. There are resources like web.dev who are actively publishing content on how to make a better future for the web.

Background

I’ve also published a few 😁 on how to improve web performance which you should totally check them out 👇🏼:

And other tips and tricks like how to use the OffscreenCanvas to render your graphics off the main thread.

This article is around the same topic, but we’re gonna see how to inspect and react to connection aspects of a visiting user to deliver a better experience to them.

There are some implementations of this in the wild such as BBC News showing a warning to users about being charged by operators if they’re on a cellular network:

Or GMail showing a ‘load basic HTML (for slow connections)’ message:

Network Information API

So far we have tried to improve the user experience by following one of the aforementioned techniques for everybody regardless of what network they are using. But what if we had a mean to inspect and detect their network connection properties and speed change while they’re browsing our applications.

That’s when the Network Information API comes into play. This API provides information about the connection such as type (i.e. WiFi or Cellular, you could see the full list 👇🏼):

Copy

enum ConnectionType {
  "bluetooth",
  "cellular",
  "ethernet",
  "mixed",
  "none",
  "other",
  "unknown",
  "wifi",
  "wimax"
};

Here is how to access this information:

Copy

let connection = navigator.connection || navigator.mozConnection || navigator.webkitConnection;

Here is how the object you’ll get looks like:

NetworkInformation object

So let’s see what are these properties one by one:

`effectiveType`

This property is showing the effective connection type. This is a bit different to the connection type we mentioned earlier. It changes based on a combination of recently observed ttl and downlink.

The possible values for effectiveType is an enum with these values:

Copy

enum EffectiveConnectionType {
  "2g",
  "3g",
  "4g",
  "slow-2g"
};

In reality if this property has slow-2g it means you should avoid sending images, videos, etc. It’s suitable for sending text only content. If the value is 2g it indicates that you could potentially send small low res images to user.

3g means you can send larger assets such as high res images, SD videos, and audio. At last 4g means the network is suited for HD video, streaming and other data heavy resources.

`downlink`

This shows the download speed of the connection. The unit uses is megabits per second.

`onchange`

This property allows you to set an event handler for the change event on NetworkInformation object.

`rtt`

This attribute represents the round trip time estimate in milliseconds nearest to 25 milliseconds and is based on application layer RTT measurements.

`saveData`

This property when getting returns true if the user has requested to reduce data usage mode from the user agent. Otherwise it returns false.

If you are wondering what happened to type property, it is not currently supported by the browsers and no one knows if that will be because the spec is in a draft mode and might get changed. You can find more information on the browser support page at the end of this article.

Use cases

Prevent from preloading videos

Now that we know what to expect from this API, lets see what we can do with it. Imagine we’re preloading videos in our page. We can hook up to the change event and stop preloading if the user is on a slower network:

Copy

let preloadVideo = true;
const slowConnections = ['2g', '2g-slow'];
var connection = navigator.connection || navigator.mozConnection || navigator.webkitConnection;
let type = connection.effectiveType;

function updateVideoPreloadingStrategy() {
  console.log("Connection type changed from " + type + " to " + connection.effectiveType);
  type = connection.effectiveType;

  if (slowConnections.containes(type)) {
    preloadVideo = false;
  }
}

connection.addEventListener('change', updateVideoPreloadingStrategy);

Load low res images

You could potentially set a class to your elements and based on that load a different image:

Copy

function updateImageLoadingStrategy() {
  console.log("Connection type changed from " + type + " to " + connection.effectiveType);
  type = connection.effectiveType;

  const myEl = document.querySelector('#id');

  if (slowConnections.containes(type)) {
    myEl.classList.add("low-res");
  } else {
    myEl.classList.remove("low-res");
  }
}

And:

Copy

#id {
  background: url(high-res.jpg) 0 0 no-repeat;
}

#id.low-res {
  background: url(low-res.jpg) 0 0 no-repeat;
}

Summary

We saw how useful can Network Information API be in most of our use cases where we want to address performance issue or even deliver a better user experience. Some other applicable use cases are:

Not load high res images
Prevent loading web fonts
HTTP calls are changed to fetch less data using summary instead of full page load
Timeout values for HTTP calls are increased
Showing warning and other informative messages to users to let them know they might get charged

💡 This API is in a draft state and might change in the future. But it doesn’t hurt to be proactive and update the implementation later rather than waiting for the full spec.

You can find more about this on the specification.

Browser support

So Network Information API is good and everything, but how about the browser support. Currently it is supported in Chrome and MS Edge only.

You can see the full list here.

Now go have fun with this API and save your users some money 😁.

Handling forms with Vue.js 🧾

Sun, 10 Nov 2019 00:00:00 GMT

Vue.js is an approachable, versatile, performant, and progressive framework to build user interfaces with. I’ve written two separate articles on it so far:

This time we will have a look at handling form with Vue.js, something which is almost inevitable in enterprise applications.

Background

When it comes to working with forms, it’s always good to know the in and out of how to handle them in your choice of framework. Of course there are some common knowledge around forms, but dealing with one can be tough at times especially if you care about the user experience.

Scenario

Assume we have a sign up form for new users on our product. We want to build this sign up form, from scratch, and step by step together.

Start

Let’s start a new application and create a component first:

Copy

<!DOCTYPE html>
<html>
<head>
  <title>Handling Forms with Vue.js</title>
  <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">

</head>

<body>
  <div class="container" id="app">
    <div class="">
      <section class="mt-md">
        <h1 class="title">Form handling with Vue.js</h1>
        <p class="text-muted">
          Learn how to work with forms, including <strong>validation</strong>!
        </p>
        <hr>      
        
        <section class="form">
          <!--Our form will be here-->
        </section>
      </section>
    </div>
  </div>
  
<script src="https://cdn.jsdelivr.net/npm/vue/dist/vue.js"></script>


<script>
  new Vue({
    el: '#app'
  })
</script>

</body>
</html>

Text inputs

As I mentioned in my intro, you can bind form input values to Vue’s data object using v-model. So let’s add a few text inputs for name and last name.

Copy

<form>
  <div class="form-group">
    <label for="firstName">First name</label>
    <input type="text" v-model="form.firstName" class="form-control" id="firstName" placeholder="Enter your name">
  </div>
  <div class="form-group">
    <label for="lastName">Last name</label>
    <input type="text" v-model="form.lastName" class="form-control" id="lastName" placeholder="Enter your last name">
  </div>
</form>

In the code snippet we just defined, we’re using two text inputs and binding them to first name and last name of the form property in the data object. The v-model creates a two way binding between our input and its corresponding property. To be sure that we’ve got everything right, you can use the code snippet below and see the values as you’re typing:

Copy

<div class="shadow">
  <div v-for="(item, k) in form">
    <strong>{{ deCamelCase(k) }}</strong> {{item}}
  </div>
</div>

And in your JavaScript code:

Copy

// the rest
<script>
  new Vue({
    el: '#app',
    data: {
      form: {
        firstName: '',
        lastName: ''
      }
    },
    methods: {
      deCamelCase: function(str) {
        const result = str.replace(/([A-Z]+)/g, " $1");
        return result.charAt(0).toUpperCase() + result.slice(1) + ':'
      }
    }
  })
</script>

If you start typing into any of those text boxes, you should see the result in the bottom div.

Setting default value

Vue will ignore the value, checked, or selected attributes of the inputs you use and uses the data object as source of truth. This means you can set default values for your form fields:

Copy

data: {
  form: {
    firstName: 'Yas',
    lastName: ''
  }
}

You can see the value of the input get set to Yas when the page renders.

Textarea

Adding a textarea is exactly like adding a normal text input:

Copy

<div class="form-group">
  <label for="notes">Additional information</label>
  <textarea v-model="form.additionalInfo" class="form-control" id="notes" rows="3"></textarea>
</div>

Don’t forget to add the additionalInfo to your data object.

Select element

Adding

Copy

<div class="form-group">
  <label for="ageGroup">Select your age group</label>
  <select v-model="form.ageGroup" class="form-control" id="ageGroup">
    <option value="1">18-22</option>
    <option value="2">22-25</option>
    <option value="3">25-27</option>
    <option value="4">27-30</option>
    <option value="5">30-33</option>
  </select>
</div>

And in your data object:

Copy

form: {
  firstName: 'Yas',
  lastName: '',
  additionalInfo: '',
  ageGroup: ''
}

If you want to add multiselect to your element, all you need to do is to add the attribute and change the type of the ageGroup property from string to array. If you want a default selection, simply set the value of the ageGroup in the data object and Vue will take of the rest.

If you want to populate the options dynamically, simply use a v-for directive:

Copy

<option v-for="option in options" v-bind:value="option.value">
  {{ option.text }}
</option>

Checkbox & Radio buttons

A single checkbox is very straightforward to use, the value of it will be a boolean:

Copy

<div class="card">
  <div class="card-body">
    <h5 class="card-title">Terms and Conditions</h5>
    <h6 class="card-subtitle mb-2 text-muted">Read and agree</h6>
    <p class="card-text">Doggo ipsum clouds what a nice floof long water shoob doggo extremely cuuuuuute,   heckin good boys long water shoob.</p>
    <div class="form-check">
      <input class="form-check-input" 
        type="checkbox" 
        v-model="form.agreeToTsCs" 
        value="" id="termsAndConditions">
      <label class="form-check-label" for="termsAndConditions">
        Agree with terms and conditions
      </label>
    </div>
  </div>
</div>

And don’t forget to add the property:

Copy

form: {
  firstName: 'Yas',
  lastName: '',
  additionalInfo: '',
  ageGroup: '',
  agreeToTsCs: false
}

For multiple checkboxes, they all use the same property, but the type would be array.

A single radio button by itself is normally useless, so you’d want to use a radio button group. The value of the property would be the value of the selected radio input:

Copy

<div class="form-check">
  <input class="form-check-input" 
    type="radio" 
    name="termSelection" 
    v-model="form.enrollingTerm"
    id="termSelection1" value="1">
  <label class="form-check-label" for="termSelection1">
    Term 1
  </label>
</div>
<div class="form-check">
  <input class="form-check-input" 
    type="radio" 
    name="termSelection" 
    v-model="form.enrollingTerm"
    id="termSelection2" 
    value="2">
  <label class="form-check-label" for="termSelection2">
    Term 2
  </label>
</div>
<div class="form-check">
  <input class="form-check-input" 
    type="radio" 
    name="termSelection" 
    v-model="form.enrollingTerm"
    id="termSelection3" 
    value="3">
  <label class="form-check-label" for="termSelection3">
    Term 3
  </label>
</div>

And add the property:

Copy

form: {
  firstName: 'Yas',
  lastName: '',
  additionalInfo: '',
  ageGroup: '1',
  agreeToTsCs: false,
  enrollingTerm: ''
}

Modifiers

There are a few modifiers which will be helpful for different scenarios and are provided out of the box. Let’s review them one by one:

.lazy

By default v-model will update the value of the input with your property on every input event. If you don’t want that to happen, you can add .lazy and it will happen after change event:

Copy

<div class="form-group">
  <label for="lazilyUpdated">This input will update after <code>change</code> event</label>
  <input type="text" class="form-control" v-model.lazy="form.lazilyUpdated" id="lazilyUpdated" placeholder="">
</div>

If you start typing on this text box, it will not get updated until you move away (blur happens).

.number

This one will cast the value to be always a number:

Copy

<div class="form-group">
  <label for="alwaysNumber">This property will always be number</code> event</label>
  <input type="number" class="form-control" v-model.number="form.alwaysNumber" id="alwaysNumber" placeholder="">
</div>

For the property you can use:

Copy

form: {
  firstName: 'Yas',
  lastName: '',
  additionalInfo: '',
  ageGroup: '1',
  agreeToTsCs: false,
  enrollingTerm: '',
  lazilyUpdated: '',
  alwaysNumber: null
}

And just to make sure it works, you can add this to your value area:

Copy

<div>{{ typeof(form.alwaysNumber) }}</div>

This is useful because input type=number will give you a string value and you have to use parseInt or parseFloat to get the correct type.

.trim

This one is very obvious, it scrapes the whitespace from typed text:

Copy

<div class="form-group">
  <label for="alwaysTrimmed">Trimmed text</code> event</label>
  <input type="text" class="form-control" v-model.number="form.alwaysTrimmed" id="alwaysTrimmed" placeholder="">
</div>

And:

Copy

form: {
  firstName: 'Yas',
  lastName: '',
  additionalInfo: '',
  ageGroup: '1',
  agreeToTsCs: false,
  enrollingTerm: '',
  lazilyUpdated: '',
  alwaysNumber: null,
  alwaysTrimmed: ''
}

Now copy paste this string into the input and watch what happens 👉🏼 test test.

Validation

You wouldn’t want to send data invalidated to your server, right? Form validation is a key to handling any form data entered by users. In its simplest form, you could implement all your validation rules in a method and check them as the user continues or even before submitting.

However, we all know how much boiler plate code you have to write for that. So the most logical option is to find out what are our options out there. The open source community is very active and for most of what you need, there is already a matured library for it. We will review the most popular choice here, vuelidate.

First you will need to install it:

Copy

npm install vuelidate --save

If you want to have validation enabled globally, you just need to import the library and use a plugin:

Copy

import Vue from 'vue'
import Vuelidate from 'vuelidate'

Vue.use(Vuelidate)

If you just want this on a single component, you could import a mixin directly:

Copy

import { validationMixin } from 'vuelidate'

var Component = Vue.extend({
  mixins: [validationMixin],
  validations: { ... }
})

Basic usage

Let’s add some validation to our form. Say we want to make the first and last name mandatory, and set a minimum length for last name.

Copy

import { required, minLength } from 'vuelidate/lib/validators'

new Vue({
  el: '#app',
  data: {
    form: {
      firstName: 'Yas',
      lastName: '',
      additionalInfo: '',
      ageGroup: '1',
      agreeToTsCs: false,
      enrollingTerm: '',
      lazilyUpdated: '',
      alwaysNumber: null,
      alwaysTrimmed: ''
    }
  },
  validations: {
    form: {
      firstName: {
        required
      },
      lastName: {
        required,
        minLength: minLength(4)
      }
    }
  },
  methods: {
    deCamelCase: function(str) {
      const result = str.replace(/([A-Z]+)/g, " $1");
      return result.charAt(0).toUpperCase() + result.slice(1) + ':'
    }
  }
})

Now in our template we need to add the validation error message:

Copy

<div class="form-group">
  <label for="firstName">First name</label>
  <input
        type="text"
        v-model="form.firstName"
        class="form-control"
        id="firstName"
        placeholder="Enter your name"
      />
  <div v-if="!$v.form.firstName.required" class="text-danger">
    First name is required</div>
</div>
<div class="form-group">
  <label for="lastName">Last name</label>
  <input
        type="text"
        v-model="form.lastName"
        class="form-control"
        id="lastName"
        placeholder="Enter your last name"
      />  
  <div v-if="!$v.form.lastName.required" class="text-danger">
    Last name is required</div>
  <div v-if="!$v.form.lastName.minLength" class="text-danger">
    Last name should be minimum 4 character</div>
</div>

Now you will have error messages shown when any of those rules are not met. Note that for first name I’ve used $error, which gives us ability to hide the error until the form is submitted or the status of the form property becomes dirty. For last name, I’ve used individual error fields, but that means that the error is visible on page load. A simple fix would be:

Copy

<div v-if="$v.form.lastName.$error">
  <div v-if="!$v.form.lastName.required"
    :class="{ 'text-danger': !$v.form.lastName.required }">
    Last name is required</div>
  <div v-if="!$v.form.lastName.minLength"
    :class="{ 'text-danger': !$v.form.lastName.minLength }">
    Last name should be minimum 4 character</div>
</div>

All the code together in action

You can find the complete code on StackBlitz and play with it to explore more scenarios.

Summary

We saw how form handling is very straight forward in Vue, and learnt a few tricks like lazy, and trim on v-model directive. We saw how error handling would be simple using a validation library instead of doing it from scratch and reinvent the wheel. And last but not least, we saw how to use vuelidate library to handle our form’s errors.

Hope this has been helpful for you and stay tuned for the next article about state management in Vue.js.

Infrastructure as code, Azure app service using a wildcard certificate from KeyVault

Thu, 31 Oct 2019 00:00:00 GMT

This post is one of those posts which is related to my consulting side. I was involved in a project where infrastructure as code was the approach from get go and since we had everything on Azure, we chose a mix of ARM templates and PowerShell. I hit a few issues along the way which I think it’s very valuable to be aware of for many people and especially myself in future on a similar project 😊.

Background

I don’t want to go through the whole architecture, suffice to know we have a few App Services and Azure Functions which need SSL binding and custom host name only in production. Since we’re managing the DNS records with Cloudflare, we got a wildcard cert for our domain and we imported the PFX file into a Key Vault in a separate resource group to be used in our release.

First try

So the ARM template we had looked like this initially:

Copy

{
  "parameters": {
    //...
  },
  "variables": {
    //...
  },
  "resources": [
    {
      "type": "microsoft.insights/components",
      "apiVersion": "2015-05-01",
      "name": "[variables('appInsightsName')]",
      "location": "[variables('appInsightsLocation')]",
      "tags": {
        "displayName": "app-insights",
        "project": "[variables('projectName')]",
        "environment": "[parameters('environment')]"
      },
      "kind": "web",
      "properties": {
        "Application_Type": "web",
        "Request_Source": "IbizaWebAppExtensionCreate"
      }
    },
    {
      "apiVersion": "2016-09-01",
      "type": "Microsoft.Web/serverfarms",
      "kind": "app",
      "name": "[variables('appServicePlanName')]",
      "tags": {
        "displayName": "app-service-plan",
        "project": "[variables('projectName')]",
        "environment": "[parameters('environment')]"
      },
      "location": "[parameters('deploymentLocation')]",
      "properties": {
        "name": "[variables('appServicePlanName')]",
        "perSiteScaling": false,
        "reserved": false,
        "targetWorkerCount": 0,
        "targetWorkerSizeId": 0
      },
      "dependsOn": [],
      "sku": {
        "name": "[parameters('appServicePlanSku')]",
        "tier": "[parameters('appServicePlanTier')]",
        "size": "[parameters('appServicePlanSize')]",
        "family": "[parameters('appServicePlanFamily')]",
        "capacity": "[parameters('appServicePlanCapacity')]"
      }
    },
    {
      "apiVersion": "2016-08-01",
      "type": "Microsoft.Web/sites",
      "kind": "app",
      "name": "[variables('appServiceName')]",
      "tags": {
        "displayName": "app-service",
        "project": "[variables('projectName')]",
        "environment": "[parameters('environment')]"
      },
      "location": "[parameters('deploymentLocation')]",
      "properties": {
        "name": "[variables('appServiceName')]",
        "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
        "siteConfig": {
          "appSettings": [
            {
              "name": "WEBSITE_RUN_FROM_PACKAGE",
              "value": "1"
            }
          ]
        },
        "httpsOnly": "[parameters('appServiceHttpsOnly')]"
      },
      "dependsOn": [
        "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
        "[resourceId('microsoft.insights/components', variables('appInsightsName'))]"
      ],
      "resources": [
        {
          "apiVersion": "2015-08-01",
          "name": "Microsoft.ApplicationInsights.AzureWebSites",
          "type": "siteextensions",
          "dependsOn": [
            "[resourceId('Microsoft.Web/Sites', variables('appServiceName'))]"
          ],
          "properties": {}
        },
        {
          "name": "appsettings",
          "type": "config",
          "apiVersion": "2015-08-01",
          "dependsOn": [
            "[resourceId('Microsoft.Web/sites', variables('appServiceName'))]",
            "Microsoft.ApplicationInsights.AzureWebSites"
          ],
          "properties": {
            "APPINSIGHTS_INSTRUMENTATIONKEY": "[reference(variables('appInsightsName'), '2015-05-01').InstrumentationKey]"
          }
        }
      ]
    },
    {
      "type": "Microsoft.Web/sites/config",
      "apiVersion": "2016-08-01",
      "name": "[concat(variables('appServiceName'), '/web')]",
      "location": "[parameters('deploymentLocation')]",
      "dependsOn": [
        "[resourceId('Microsoft.Web/sites', variables('appServiceName'))]"
      ],
      "tags": {
        "displayName": "app-service",
        "project": "[variables('projectName')]",
        "environment": "[parameters('environment')]"
      },
      "properties": {
        "alwaysOn": "[parameters('appServiceAlwaysOn')]"
      }
    },
    {
      "type": "Microsoft.Web/certificates",
      "name": "[variables('certificateName')]",
      "apiVersion": "2016-03-01",
      "location": "[resourceGroup().location]",
      "condition": "[equals(parameters('environment'), 'prod')]",
      "properties": {
        "keyVaultId": "[parameters('prodKeyVaultId')]",
        "keyVaultSecretName": "[parameters('prodKeyVaultSecretName')]",
        "serverFarmId": "[resourceId('Microsoft.Web/serverFarms', variables('appServicePlanName'))]"
      },
      "dependsOn": ["[concat('Microsoft.Web/sites/', variables('appServiceName')]"]
    },
    {
      "type": "Microsoft.Web/sites/hostnameBindings",
      "name": "[concat(variables('appServiceName'), '/', parameters('customHostname'))]",
      "apiVersion": "2016-03-01",
      "location": "[resourceGroup().location]",
      "condition": "[equals(parameters('environment'), 'prod')]",
      "properties": {
        "sslState": "SniEnabled",
        "thumbprint": "[reference(resourceId('Microsoft.Web/certificates', variables('certificateName'))).Thumbprint]"
      },
      "dependsOn": [
        "[concat('Microsoft.Web/certificates/', variables('certificateName'))]"
      ]
    }
  ],
  "outputs: {
    //...
  }

}

There are a few important things here you need to pay attention to. First, we have the Microsoft/certificate conditionally deployed, same as sites/hostnameBindings if we’re deploying to production. Second, note how we have a reference to the certificate in Key Vault in the certificate properties. And last, we have a reference to the certificate in the hostNameBinding in the same template.

First try

When we first tried this, we hit a permission error on the Key Vault the template was referencing. I had made sure that the Azure DevOps service principal is added into the access policy of the Key Vault and also the tick the Azure Resource Manager for template deployment checkbox to allow referencing the secret from within ARM template.

Error Failed to add App Service certificate to the app, Check error for more details. Error Details: The service does not have access to ‘/subscriptions/XXXebXXX-XXX-XXXX-XXX-XXXXX/resourcegroups/XXXXX/providers/microsoft.keyvault/vaults/xxxxxxxxvault’ Key Vault. Please make sure that you have granted necessary permissions to the service to perform the request operation.

This was really strange to me and I had no clue as to what’s missing. However, after a bit of Googling, I found out there are two service principals which belong to Microsoft and you have to add those to your Key Vault’s access policy in order to be able to link a certificate to your App Services.

Here is the kick starter template repository on GitHub which explains what needs to be done. But in short, you need to run these commands:

Copy

Login-AzureRmAccount

Set-AzureRmContext -SubscriptionId AZURE_SUBSCRIPTION_ID

Set-AzureRmKeyVaultAccessPolicy -VaultName KEY_VAULT_NAME -ServicePrincipalName f3c21649-0979-4721-ac85-b0216b2cf413 -PermissionsToSecrets get,set,delete

Set-AzureRmKeyVaultAccessPolicy -VaultName KEY_VAULT_NAME -ServicePrincipalName abfa0a7c-a6b6-4736-8310-5855508787cd -PermissionsToSecrets get

After running these commands, the error was gone and we could progress further. Note that the application ids are fixed.

Second try

Once the necessary permissions was granted, we run the deployment again to be hit with the second issue, this time I was so baffled by this error:

Error Another certificate exists with same thumbprint XXXXXXXXXXXXXXXXXXXX at location xxxx in the Resource Group xxxxxx.

I wasn’t sure what’s causing the issue since I thought that since we’re reusing the same certificate, linking them wouldn’t be a problem. But when I looked at the deployments on the resource group, I found out Azure is trying to create multiple Microsoft•Web/certificates resources which will have the same thumbprint and so it fails. As Aussies say it, fair dinkum.

So I extracted the certificate resource to the parent ARM template and passed a reference of it down to linked templates:

Copy

{
  //parent ARM template
  "resources": [
    {
      "type": "Microsoft.Web/certificates",
      "name": "[parameters('prodKeyVaultSecretName')]",
      "condition": "[equals(parameters('environment'), 'prod')]",
      "apiVersion": "2016-03-01",
      "location": "[parameters('deploymentLocation')]",
      "properties": {
        "keyVaultId": "[parameters('prodKeyVaultId')]",
        "keyVaultSecretName": "[parameters('prodKeyVaultSecretName')]"
      }
    },
    //...
    {
      "apiVersion": "2017-05-10",
      "name": "web-client-app-service-linked-template",
      "type": "Microsoft.Resources/deployments",
      "metadata": {
        "comments": "Linked ARM template deploys web client app service"
      },
      "properties": {
        "mode": "Incremental",
        "templateLink": {
          "uri": "[concat('https://', parameters('artefactsStorageAccountName'),'.blob.core.windows.net/',parameters('artefactsStorageAccountContainerName'),'/client-web-app/azuredeploy.json',parameters('artefactsStorageAccountSasToken'))]",
          "contentVersion": "1.0.0.0"
        },
        "parametersLink": {
          "uri": "[concat('https://', parameters('artefactsStorageAccountName'), '.blob.core.windows.net/', parameters('artefactsStorageAccountContainerName'),'/client-web-app/azuredeploy.parameters-', parameters('environment'), '.json', parameters('artefactsStorageAccountSasToken'))]",
          "contentVersion": "1.0.0.0"
        }
      },
      "dependsOn": [
        "[resourceId('Microsoft.Web/certificates', parameters('prodKeyVaultSecretName'))]"
      ]
    },
  ]
}

And in the linked template:

Copy

{
  "resources": [
    {
      "apiVersion": "2016-08-01",
      "type": "Microsoft.Web/sites",
      "kind": "app",
      "name": "[variables('appServiceName')]",
      "tags": {
        "displayName": "app-service",
        "project": "[variables('projectName')]",
        "environment": "[parameters('environment')]"
      },
      "location": "[parameters('deploymentLocation')]",
      "properties": {
        "name": "[variables('appServiceName')]",
        "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
        "siteConfig": {
          "appSettings": [
            {
              "name": "WEBSITE_RUN_FROM_PACKAGE",
              "value": "1"
            }
          ]
        },
        "httpsOnly": "[parameters('appServiceHttpsOnly')]",
        "hostNameSslStates": [
          {
            "name": "[parameters('customHostname')]",
            "sslState": "SniEnabled",
            "thumbprint": "[reference(concat('Microsoft.Web/certificates/', parameters('certificateName')), '2016-03-01').Thumbprint]",
            "toUpdate": true
          }
        ]
      },
      "dependsOn": [
        "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
        "[resourceId('microsoft.insights/components', variables('appInsightsName'))]"
      ],
      "resources": [
        {
          "apiVersion": "2015-08-01",
          "name": "Microsoft.ApplicationInsights.AzureWebSites",
          "type": "siteextensions",
          "dependsOn": [
            "[resourceId('Microsoft.Web/Sites', variables('appServiceName'))]"
          ],
          "properties": {}
        },
        {
          "name": "appsettings",
          "type": "config",
          "apiVersion": "2015-08-01",
          "dependsOn": [
            "[resourceId('Microsoft.Web/sites', variables('appServiceName'))]",
            "Microsoft.ApplicationInsights.AzureWebSites"
          ],
          "properties": {
            "APPINSIGHTS_INSTRUMENTATIONKEY": "[reference(variables('appInsightsName'), '2015-05-01').InstrumentationKey]"
          }
        },
        {
          "type": "hostNameBindings",
          "name": "[parameters('customHostname')]",
          "condition": "[equals(parameters('environment'), 'prod')]",
          "apiVersion": "2016-08-01",
          "dependsOn": [
            "[resourceId('Microsoft.Web/sites', variables('appServiceName'))]"
          ],
          "properties": {
            "siteName": "[variables('appServiceName')]",
            "domainId": null,
            "hostNameType": "Verified"
          }
        }
      ]
    },
  ]
}

Notice the reference to the certificate resource in the hostNameSslStates property. And again we kicked off a new release to be hit with another problem.

This time it complained about a missing resource for non prod environments which was fair, as I forgot to apply the condition to the thumbprint reference.

Third try

So I applied an if condition to the thumbprint thinking it would work:

Copy

{
  "hostNameSslStates": [
    {
      "name": "[parameters('customHostname')]",
      "sslState": "SniEnabled",
      "thumbprint": "[if(equals(parameters('environment'), 'prod'), reference(concat('Microsoft.Web/certificates/', parameters('certificateName')), '2016-03-01').Thumbprint, '')]",
      "toUpdate": true
    }
  ]
}

Which failed again. This time I found out that the reference evaluation happens regardless of if statement. So I removed the hostNameSslStates altogether and used the full resource:

Copy

{
  "resources": [
    {
      "type": "hostNameBindings",
      "name": "[parameters('customHostname')]",
      "condition": "[variables('isProd')]",
      "apiVersion": "2016-08-01",
      "dependsOn": [
        "[resourceId('Microsoft.Web/sites', variables('appServiceName'))]"
      ],
      "properties": {
        "siteName": "[variables('appServiceName')]",
        "domainId": null,
        "hostNameType": "Verified",
        "sslState": "SniEnabled",
        "thumbprint": "[reference(concat('Microsoft.Web/certificates/', parameters('certificateName')), '2016-03-01').Thumbprint]"
      }
    }
  ]
}

Which failed again 😭😭😭. For the same reason as before, which was the reference to the missing resource.

Forth try

So I tried the if condition trick again this time on the thumbprint property of the hostNameBindings resource:

Copy

{
  "type": "hostNameBindings",
  "name": "[parameters('customHostname')]",
  "condition": "[variables('isProd')]",
  "apiVersion": "2016-08-01",
  "dependsOn": [
    "[resourceId('Microsoft.Web/sites', variables('appServiceName'))]"
  ],
  "properties": {
    "siteName": "[variables('appServiceName')]",
    "domainId": null,
    "hostNameType": "Verified",
    "sslState": "SniEnabled",
    "thumbprint": "[if(variables('isProd'), reference(concat('Microsoft.Web/certificates/', parameters('certificateName')), '2016-03-01').Thumbprint, '')]"
  }
}

And guess what, it was successful. OMG, I was so happy that this was working.

Summary

So if you have a wildcard certificate and you want to reuse it in multiple App Services, DO NOT create the certificate for one of those, simply create one and share it with all of them.

Here are some points which might be helpful to know:

The ARM deployment respects the conditional resource in the dependsOn section. Meaning if the resource is conditionally created, it doesn’t have any effect in the order of deployment and it won’t fail it either.
You can’t use a reference function in the variable section. Otherwise my life would’ve been much easier 😁.
When using a reference on a property, the function is evaluated regardless of the fact that you have an if condition or not (happy to be proven wrong with this one).
And last, you need to be patient when working with ARM templates 😂.

Hope this article helps someone else and saves their time. I am planning to create a template set and create a PR into kick starter repo. But don’t have any timeline as of now.

Yashints | Blog

🖥️ Use local AI to supercharge your coding skills

Why bother?

Setup - let’s get cracking

Install the code generation model

Install the extension

Configuring the extension

Testing

Using tab for auto completion

Refactor code

Ask questions about your codebase

Use documentation as context for your questions

Perform actions with slash

Use files as context

Understand errors in terminal

Summary

🔮 fabric, augmenting humans using AI

The why

Introducing fabric

Prerequisites

Installing fabric

Features

Taking it to next level

What next

Summary

🥇 Run your generative AI model locally 🥇

🩺 The need

🖥️ Prerequisites

⚙️ Installation

Running as service

📱 User interface

🤖 Custom model

✍🏽 Summary

Semantic kernel series - Last part - Planners

Native function plugins

Planner

Creating the planner

Automatic function calling

Summary

Semantic kernel series - Part two - Plugins

Context

Setup

Prompts

Plugin setup

Prompt text

Input

Setting up the kernel

Output

What next?

Semantic kernel series - Intro

Semantic Kernel’s components

A little taste of what to expect

A simple quick start

What’s next?

Microsoft Graph SDK has changed, a lot!!!

Context

Setup

Creating the client

Removal of the request method

Adding header and url parameters

Handing the response

Error handling

Other cool features

So, I decided to upgrade my blog's engine!!!

The why

Updating Node and packages

Updating GraphQL queries

Migration from gatsby-image

Changes to the way we add SEO

Styled component updates

Summary

My first ever book on Azure Bicep

Why would you want to write a book?

Sharing your knowledge with others

It would add to your credit professionally

Ticking a box, accomplishing a new goal

You will learn a lot

You might make money

The HOW

Finding a topic

Removal of the `request` method

Migration from `gatsby-image`

The `any` keyword