Docker announces collaboration with Azure

May 28, 2020 iamashishsharma1 Comment

Docker today announced that it has extended its strategic collaboration with Microsoft to simplify code to cloud application development for developers and development teams by more closely integrating with Azure Container Instances (ACI).

Docker and Microsoft’s collaboration extends to other products as well. Docker will add integrations for Visual Studio Code, Microsoft’s widely used free code editor, to let developers who use the tool deploy their code as containers faster. After this collaboration, you can login directly to Azure from Docker CLI, then you can select Docker context. If we have resource group, you can select it or you can create a new resource group. Then, you can run individual or multiple containers using Docker Compose.

docker login Azure

docker context create aci-westus aci --aci-subscription-id xxx --aci-resource-group yyy --aci-location westus

docker context use aci-westus

Tighter integration between Docker and Microsoft developer technologies provides the following productivity benefits:

Easily log into Azure directly from the Docker CLI
Trigger an ACI cloud container service environment to be set up automatically with easy to use defaults and no infrastructure overhead
Switch from a local context to a cloud context to run applications quickly and easily
Simplifies single container and multi-container application development via the Compose specification allowing a developer to invoke fully Docker compatible commands seamlessly for the first time natively within a cloud container service
Provides developer teams the ability to share their work through Docker Hub by sharing their persistent collaborative cloud development environments where they can do remote pair programming and real-time collaborative troubleshooting

For more information & developers can sign up for Docker Desktop & VS Code beta here.

Azure Front Door Deployment using ARM Template

May 3, 2020 iamashishsharma5 Comments

In this article, we will provision Azure Front Door using Arm Template. I have already created two Azure Web Apps which are running in two different regions i.e West Europe & South East Asia region. We will create & configure Azure Front Door to direct the traffic to these web apps.

Follow the below steps to deploy Azure Front Door:

Download this git repo and fill the parameters in azuredeploy.parameters.json as shown below:

{
  \"$schema\": \"https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#\",
  \"contentVersion\": \"1.0.0.0\",
  \"parameters\": {
    \"frontDoorName\": {
      \"value\": \"ashishfrontdoortest\"
    },
    \"dynamicCompression\": {
      \"value\": \"Enabled\"
    },
    \"backendPools1\": {
      \"value\": {
        \"name\": \"backendpool1\",
        \"backends\": [
          {
            \"address\": \"ashishtest1.azurewebsites.net\",
            \"httpPort\": 80,
            \"httpsPort\": 443,
            \"weight\": 50,
            \"priority\": 1,
            \"enabledState\": \"Enabled\"
          },
          {
            \"address\": \"ashishtest2.azurewebsites.net\",
            \"httpPort\": 80,
            \"httpsPort\": 443,
            \"weight\": 50,
            \"priority\": 2,
            \"enabledState\": \"Enabled\"
          }
        ]
      }
    }
  }
}

Run the below PowerShell command to provision the AFD.

Login-AzureRmAccount
$ResourceGroupName = \"\" #Enter Resource group Name
$AzuredeployFileURL=\"\" #File path of azuredeploy.json
$AzuredeployParametersFile = \"\" #File path of azuredeploy.parameters.json
Set-AzureRmContext -SubscriptionId \"\" #Subscription ID
New-AzureRmResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateFile $AzuredeployFileURL -TemplateParameterFile $AzuredeployParametersFile

Once Azure front Door is deployed, go to browser and navigate the Frontend host URL of AFD. Your request will be routed to Azure Web App.

Go to Azure Portal to check the AFD configuration. Click on the Front Door Designer from left navigation bar and open Frontends/domains.

Here you can enable SESSION AFFINITY & WEB APPLICATION FIREWALL

Now open backend pools as illustrated below. Here you can update your backend pools configuration.

Last section is routing rule which maps your frontend host and a matching URL path pattern to a specific backend pool.

Here you can configure URL rewrite, Caching, Dynamic compression, Query String caching behavior etc.

Difference between Azure Front Door Service and Traffic Manager

April 15, 2020 iamashishsharmaLeave a comment

Azure Front Door Service is Microsoft’s highly available and scalable web application acceleration platform and global HTTP(s) load balancer. Azure Front Door Service supports Dynamic Site Acceleration (DSA), SSL offloading and end to end SSL, Web Application Firewall, cookie-based session affinity, URL path-based routing, free certificates and multiple domain management.

In this article, I will compare Azure Front Door to Azure Traffic Manager in terms of performance and functionality.

Similarity:

Azure Front Door service can be compared to Azure Traffic Manager in a way that this also provides global HTTP load balancing to distribute traffic across different Azure regions, cloud providers or even with your on-premises.

Both AFD & Traffic Manager support:

Multi-geo redundancy: If one region goes down, traffic routes to the closest region without any intervention.

Closest region routing: Traffic is automatically routed to the closest region.

Differences:

Azure Front Door provides TLS protocol termination (SSL offload), and Azure Traffic Manager does not. It means AFDs take load off from the Web Front Ends, which do not have to encrypt or decrypt the request.

Azure Front Door provides application layer processing, and Azure Traffic Manager does not.

While using AFS, user will experience better performance than traffic manager as AFD uses Anycast, which provides lower latency, thereby providing higher performance.

AFD provides WAF feature for your application to provide security from DDoS attacks.

We can perform URL rewriting in Azure Front Door but not in Traffic Manager.

Traffic manager relies on DNS lookup for network routing while AFD uses reverse proxy which provides faster failover support.

AFD caches the static content while no caching mechanism is available in Traffic Manager.

Quick Summary:

Azure Front Door	Traffic Manager
Cross region redirection & availability.	Cross region redirection & availability.
SSL offload	No SSL offload
Uses Reverse proxy	Uses DNS lookup
Caches static content	No caching available
Uses AnyCast & Split TCP	Does not use such service
AFD provides WAF feature	No such feature
99.99% SLA	99.99% SLA

I hope this information helped you. In our next article, we will discuss how to create and use Azure Front Door service.

Deploy Azure CDN in existing Sitecore environment using Arm Template

March 29, 2020 iamashishsharma3 Comments

In this article, we will deploy Azure CDN in existing Sitecore XP environment using Powershell & Arm template.

Refer the below steps to deploy Azure CDN:

1. Download Azure CDN WDP from dev.sitecore.netand upload the WDP in Azure Storage blob container.

Once it is uploaded, you will find two topology XP & XM. I have selected CDN XP topology for this demo. Then, create a SAS token & save it.

2. Create a parameters.json file as shown below:

{
  \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#\",
  \"contentVersion\": \"1.0.0.0\",
  \"parameters\": {
    \"cdWebAppName\": {
      \"value\": \"<>\"
    },
    \"cdnSKU\": {
      \"value\": \"<>\"
    },
    \"cdnMsDeployPackageUrl\": {
      \"value\": \"<>\"
    }
  }
}

Fill the ARM template parameter

cdWebAppName	Name of CD instance
cdnSKU	Premium_Verizon, Standard_Verizon, Standard_Akamai, Standard_Microsoft
cdnMsDeployPackageUrl	SAS token of CDN WDP

3. Run the below PowerShell command:

Login-AzAccount

$ResourceGroupName = \"<<Enter Resource group name"

$AzuredeployFileURL=\"https://raw.githubusercontent.com/Sitecore/Sitecore-Azure-Quickstart-Templates/master/CDN%201.0.0/xp/azuredeploy.json\"

$AzuredeployParametersFile = \"<<path of above created parameters.json file"

Set-AzContext -Subscription “<>”

New-AzResourceGroupDeployment -ResourceGroupName $ResourceGroupName -TemplateUri $AzuredeployFileURL -TemplateParameterFile $AzuredeployParametersFile -Verbose

4. Once the script is executed successfully, go to CD instance & you can see CDN.config file inside AppConfig/Include/CDN folder which consists of below settings:

Media.AlwaysIncludeServerUrl
Media.MediaLinkServerUrl
Media.AlwaysAppendRevision
MediaResponse.Cacheability
MediaResponse.MaxAge

Read this article to know more about these settings: https://doc.sitecore.com/developers/92/sitecore-experience-manager/en/sitecore-media-library-cdn-related-configuration-reference.html

Navigate your CD instance and inspect any image in your web page, you can see the image URL consists of CDN URL.

For more details about CDN setup in Sitecore refer the below articles:

https://doc.sitecore.com/developers/92/sitecore-experience-manager/en/enabling-cdn.html

https://doc.sitecore.com/developers/92/sitecore-experience-manager/en/manually-configure-the-sitecore-media-library-to-use-a-cdn.html

https://doc.sitecore.com/developers/93/sitecore-experience-manager/en/cdn-setup-considerations.html

Kubernetes Service Discovery

January 26, 2020 iamashishsharmaLeave a comment

Service discovery solves the problem of figuring out which process is listening on which address/port for which service.

In a good service discovery system:

users can retrieve info quickly
informs users of changes to services
low latency
richer definition of what the service is (not sure what this means)

The Service Object

a way to create a named label selector
created using kubectl expose
Each service is assigned a virtual IP called cluster IP – the system load balances across this IP all the pods identified by the same selector

Kubernetes itself creates and runs a service called kubernetes which lets the components in your app talk to other components such as the API server.

Service DNS

k8s inbuilt DNS service maps cluster IPs to DNS names
installed as a system component when the cluster is created, managed by k8s
Within a namespace, any pod belonging to a service can be targeted just by using the service name

Readiness Checks

The Service objects also tracks when your pods are ready to handle requests.

spec:

…

template:

…

spec:

containers:

…

name: alpaca-prod

readinessProbe:

httpGet:

path: /ready

port: 8080

periodSeconds: 2

initialDelaySeconds: 0

failureThreshold: 3

successThreshold: 1

I we add the readinessProbe section to our YAML for the deployment, then the pods created by this deployment will be checked at the /ready endpoint on port 8080. As soon as the pod comes up, we hit that end point every 2 seconds. If one check succeeds, the pod will be considered ready. However, if three checks fail in succession, it’ll not be considered ready. Requests from you application are only sent to pods that are ready.

Looking Beyond the Cluster

To allow traffic from outside the cluster to reach it, we use something known as NodePorts.

This feature assigns a specific port to the service along with the cluster IP
Whenever any node in this cluster receives a request on this port, it automatically forwards it to the service
If you can reach any node in the cluster, you can reach the service too, without knowing where any of its pods are running

Azure load-balancing options

January 6, 2020 iamashishsharmaLeave a comment

Load balancing refers to the process of distributing incoming network traffic uniformly across a group of backend server also known as server farm to optimize network efficiency, reliability and capacity.

Microsoft provides a good documentation which describes load-balancing options in Azure. Here is a quick summary of load-balancing options in Azure.

Options	Azure Front Door	Traffic Manager	Application Gateway	Azure Load Balancer
Description	It offers Layer 7 capabilities for your application like SSL offload, path-based routing, fast failover, caching, etc. to improve performance and high-availability of your applications.	DNS-based traffic load balancer that enables you to distribute traffic optimally to services across global Azure regions, while providing high availability and responsiveness.	Provides application delivery controller (ADC) as a service, offering various Layer 7 load-balancing capabilities.	Zone-redundant, high-performance, low-latency Layer 4 load-balancing service (inbound and outbound) for all UDP and TCP protocols.
OSI Layer	Layer 7 (Application Layer)		Layer 7 (Application Layer)	Layer 4 (Transport Layer)
Global/regional	Global	Global	Regional	Regional
Traffic type	HTTP(S)	non-HTTP(S)	HTTP(S)	non-HTTP(S)
SLA	99.99%	99.99%	99.95%	99.99%
Routing	Reverse proxy. AFD uses reverse proxy which provides faster failover support. Also uses AnyCast & Split TCP.	DNS routing. DNS-based load-balancing service that operates only at the domain level.	It acts as a reverse proxy service. This terminates the client connection and forwards request to back endpoints.	This provides network level distribution but essentially only with the same azure data centre.
SSL Offload	Available		Available
WAF features	Available		Available
Caching	Available

The following flowchart is really helpful to identify best load balancing option for your application.

afdtmwaflb

Source: https://docs.microsoft.com/en-us/azure/architecture/guide/technology-choices/load-balancing-overview

Kubernetes DaemonSet

December 23, 2019 iamashishsharmaLeave a comment

DaemonSets

replicates a single pod on all or a subset of nodes in the cluster
land an agent or daemon on every node for, say, logging, monitoring, etc.

ReplicaSet and DaemonSet both create pods which are expected to be long-running services & try to match the observed and desired state in the cluster.

Use a ReplicaSet when:

the app is completely decoupled from the node
multiple copies of the pod can be run on one node
no scheduling restrictions need to be in place to ensure replicas don\’t run on the same node

Use a DaemonSet when:

one pod per node or subset of nodes in the cluster

DaemonSet Scheduler

By default the a copy is created on every pod
The nodes can be limited using node selectors which matches to a set of labels
DaemonSets determine which node a pod will run on when creating it using the nodeName field
Hence, the k8s scheduler ignores the pods created by DaemonSets

The DaemonSet controller:

creates a pod on each node that doesn\’t have one
if a new node is added to the cluster, the DaemonSet controller adds a pod to it too
it tries to reconcile the observed state and the desired state

Creating DaemonSets

the name should be unique across a namespace
includes a pod spec
creates the pod on every node if a node selector isn\’t specified

Limiting DaemonSets to Specific Nodes

Add labels to nodes

$ kubectl label nodes k0-default-pool-35609c18-z7tb ssd=true

node \”k0-default-pool-35609c18-z7tb\” labeled

Add the nodeSelector key in the pod spec to limit the number of nodes the pod will run on:

apiVersion: extensions/v1beta1

kind: \”DaemonSet\”

metadata:

labels:

app: nginx

ssd: \”true\”

name: nginx-fast-storage

spec:

template:

metadata:

labels:

app: nginx

ssd: \”true\”

spec:

nodeSelector:

ssd: \”true\”

containers:

– name: nginx

image: nginx:1.10.0

If the labels specified as a nodeSelector is added to a new to existing node, the pod will be created on that node. Similarly, if the label is removed from an existing node, the pod will also be removed.

Updating a DaemonSet

For k8s < 1.6, the DaemonSet was updated and the pods were deleted manually
For k8s >= 1.6, the DaemonSet has an equivalent to the Deployment object which manages the rollout

Updating a DaemonSet by Deleting Individual Pods

manually delete the pods associated with a DaemonSet
delete the entire DaemonSet and create a new one with the updated config. This approach causes downtime as all the pods associated with the DaemonSet are also deleted.

Rolling Update of a DaemonSet

for backwards compatibility, the default update strategy is the one described above
to use a rolling update set the following in your yaml: spec.updateStrategy.type: RollingUpdate
any change to spec.template or sub-field will initiate a rolling update
it\’s controlled by the following two params:

spec.minReadySeconds, which determines how long a Pod must be “ready” before the rolling update proceeds to upgrade subsequent Pods
spec.updateStrategy.rollingUpdate.maxUnavailable, which indicates how many Pods may be simultaneously updated by the rolling update.

A higher value for spec.updateStrategy.rollingUpdate.maxUnavailable increases the blast radius in case a failure happens but increase the time it takes for the rollout.

Note: In a rolling update, the pods associated with the DaemonSet are upgraded gradually while some pods are still running the old configuration until all pods have the new configuration.

To check the status of the rollout, run: kubectl rollout

Deleting a DaemonSet

kubectl delete -f daemonset.yml

Deleting a DaemonSet deletes all the pods associated with it
in order to retain the pods and delete only the DaemonSet, use the flag: –cascade = false

Basic Kubectl Commands

November 19, 2019 iamashishsharmaLeave a comment

Some of the basic kubectl commands apply to all k8s objects

Namespaces

used to group objects in the cluster
each namespace acts like a folder holding a set of objects
kubectl works with the default namespace by default
—namespace can be passed to specify a different one

Context

can be used to change the default namespace, manage different clusters or different users for authenticating to them
To change the namespace to abc, run: $ kubectl config set-context my-context –namespace=abc
This command just creates a context. To run it, use: $ kubectl config use-context my-context
This records the change in the kubectl config file while is usually located at $HOME/.kube/config. This file also stores information related to finding and authenticating to the cluster.

Viewing Kubernetes API Objects

Everything is represented by a RESTful resource, called objects.
Each object has a unique HTTP endpoint scoped using namespaces.
kubectl uses these endpoints to fetch the objects
Viewing resources in the current namespace: kubectl get
Viewing a specific resource: kubectl get Use describe instead of `get“ for more detailed information about the object
To get more info than normally displayed, use -o wide flag
To get raw JSON or YAML objects from the API server, use -o json, -o yaml.

Creating, Updating, and Destroying Kubernetes Objects

k8s objects are represented using YAML or JSON files (each object has its own JSON/YAML file?)
used to manipulate objects on the server
To create an object described using obj.yaml: kubectl apply -f obj.yaml K8s automatically infers the type, it doesn’t need to be specified.

Labeling and Annotating Objects

used to tag objects
kubectl label/annotate pods bar color=red adds the color=red label to a pod called bar.
To remove a label: kubectl label pods bar -color

Debugging Commands

To see logs for a container: kubectl logs

Use -c to choose a particular container
Use -f to stream logs to terminal
kubectl exec -it — bash provides an interactive shell within the container.

Kubernetes ReplicaSet

October 23, 2019 iamashishsharmaLeave a comment

We need multiple replicas of containers running at a time because:

Redundancy: fault-tolerant system
Scale: more requests can be served
Sharding: computation can be handled in a parallel manner

Multiple copies of pods can be created manually, but it’s a tedious process. We need a way in which a replicated set of pods can be managed and defined as a single entity. This what the ReplicaSet does: ensures the right types and number of pods are running correctly. Pods managed by a ReplicaSet are automatically rescheduled during a node failure or network partition.

When defining ReplicaSet, we need:

specification of the pods we want to create
desired number of replicas
a way of finding the pods controlled by the ReplicaSet

Reconciliation Loops

This is the main concept behind how ReplicaSets work and it is fundamental to the design and implementation of Kubernetes. Here we deal with two states:

desired state is the state you want – the desired number of replicas
the current state is the observed staten the present moment – the number of pods presently running
The reconciliation loop runs constantly to check if there is a mismatch between the current and the desired state of the system.
If it finds a mismatch, then it takes the required actions to match the current state with what’s desired.
For example, in the case of replicating pods, it’ll decide whether to scale up or down the number of pods based on what’s specified in the pod’s YAML. If there are 2 pods and we require 3, it’ll create a new pod.

Benefits:

goal-driven
self-healing
can be expressed in few lines of code

Relating Pods and ReplicaSets

pods and ReplicaSets are loosely coupled
ReplicaSets doesn’t own the pods they create
use label queries to identify which set of pods they’re managing

This decoupling supports:

Adopting Existing Containers:

If we want to replicate an existing pod and if the ReplicaSet owned the pod, then we’d have to delete the pod and re-create it through a ReplicaSet. This would lead to downtime. But since they’re decoupled, the ReplicaSet can simply “adopt” the existing pod.

Quarantining Containers

If a pod is misbehaving and we want to investigate what’s wrong, we can isolate it by changing its labels instead of killing it. This will dissociate it from the ReplicaSet and/or service and consequently the controller will notice that a pod is missing and create a new one. The bad pod is available to developers for debugging.

Designing with ReplicaSets

represent a single, scalable microservice in your app
every pod created is homogeneous and are inter-changable
designed for stateless services

ReplicaSet spec

must’ve a unique name (within a namespace or cluster-wide?)
a spec section that contains:

number of replicas
pod template

Pod Templates

created using the pod template in the spec section
the ReplicaSet controller creates & submits the pod manifest to the API server directly

Labels

ReplicaSets use labels to filter pods it’s tracking and responsible for
When a ReplicaSet is created, it queries the API server for the list of pods, filters it by labels. It adds/deletes pods based on what’s returned.

Scaling ReplicaSets

Imperative Scaling

kubectl scale replica-set-name —replicas=4

Don’t forget to update any text-file configs you’ve to match the value set imperatively.

Declarative Scaling

Change the replicas field in the config file via version control and then apply it to the cluster.

Autoscaling a ReplicaSet

K8s has a mechanism called horizontal pod autoscaling (HPA). It is called that because k8s differentiates between:

horizontal scaling: create additional replicas of a pod
vertical scaling: adding resources (CPU, memory) to a particular pod

HPA uses a pod known as heapster in your cluster to work correctly. This pod keeps track of metrics and provides an API for consuming those metrics when it makes scaling decisions.

Note: There is no relationship between HPA and ReplicaSet. But it’s a bad idea to use both imperative/declarative management and autoscaling together. It can lead to unexpected behaviour.

Deleting ReplicaSets

Deleting a ReplicaSet deletes all the pods it created & managed as well. TO delete only the ReplicaSet object & not the pods, use —cascasde=false.

Deploy docker image to Azure Kubernetes Service

October 15, 2019 iamashishsharmaLeave a comment

In this tutorial, we will learn how to deploy container image into Kubernetes cluster using Azure Kubernetes Service. I am assuming that you have already pushed an image into your Azure container registry.

You may refer the below articles for Docker image & Azure container registry.

Create Container image

Push container image to Azure container registry

NOTE: We will use Azure CLI hence install Azure CLI version 2.0.29 or later. Run az –version to find the version.

Select your subscription. Replace <> with your subscription id.

az account set --s <>

2. If you have an existing resource group, you can skip this step else you can create a new resource group.

az group create --name <> --location <>

Example:

az group create --name helloworldRG --location westeurope

Create Azure Kubernetes Service.

az aks create --resource-group <> --name <> --node-count <> --enable-addons monitoring --generate-ssh-keys --node-resource-group <>

Example:

az aks create --resource-group helloworldRG --name helloworldAKS2809 --node-count 1 --enable-addons monitoring --generate-ssh-keys --node-resource-group helloworldNodeRG

While creating an AKS cluster, one more resource group is also created to store AKS resource. For more details, refer Why are two resource groups created with AKS?

Connect to AKS cluster

az aks get-credentials --resource-group <>  --name <>

Example:

az aks get-credentials --name \"helloworldAKS2809\" --resource-group \"helloworldRG\"

To verify your connection, run the below command to get nodes.

kubectl get nodes

Integrate AKS with ACR.

az aks update -n myAKSCluster -g myResourceGroup --attach-acr

Example

az aks update -n helloworldAKS2809 -g helloworldRG --attach-acr helloworldACR1809

Deploy image to AKS.

Create a file hellowordapp.yaml and copy the below content in the file.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworldapp
  labels:
    app: helloworldapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: helloworldapp
  template:
    metadata:
      labels:
        app: helloworldapp
    spec:
      containers:
      - name: hellositecore2705
        image: helloworldacr1809.azurecr.io/helloworldapp:latest
        ports:
        - containerPort: 8080
apiVersion: v1
kind: Service
metadata:
  name: helloworldapp
spec:
  selector:
    app: helloworldapp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: helloworldapp

You can refer Yaml validator to check your yaml file.

It’s time to deploy our docker image from ACS to AKS.

kubectl apply -f <>

Example:

kubectl apply -f hellowordapp.yaml

To get the external IP run the command.

kubectl get service helloworldapp

aks2

Open the web Brower and navigate to your external IP. The web page will be opened as shown below: aks3

I hope this information helped you. If you have any feedback, questions or suggestions for improvement please let me know in the comments section.

Kubernetes APIs and Access

September 27, 2019 iamashishsharmaLeave a comment

The entire Kubernetes architecture is API-driven, the main agent for communication (internal and external) is the Kubernetes-apiserver. There are API groups that may have multiple versions and follow a domain-name format with reserved names such as the empty group and names ending in .k8s.io.

View the API groups with a curl query:

$ curl https://127.0.0.1:6443/apis -k

….

{

\”name\”: \”apps\”,

\”versions\”: [

{

\”groupVersion\”: \”apps/v1beta1\”,

\”version\”: \”v1beta1\”

{

\”groupVersion\”: \”apps/v1beta2\”,

\”version\”: \”v1beta2\”

}

….

Make the API calls with kubectl (recommended) or use curl or other program providing the certificates, keys, and JSON string or file when required.

curl –cert userbob.pem \\

–key userBob-key.pem \\

–cacert /path/to/ca.pem \\

https://k8sServer:6443/api/v1/pods

It\’s important to check authorizations. Use kubectl to check authorizations as administrator and as a regular user (i.e. bob) in different namespaces:

$ kubectl auth can-i create deployments

yes

$ kubectl auth can-i create deployments –as bob

$ kubectl auth can-i create deployments –as bob –namespace developer

yes

There are 3 APIs which can be applied to set who and what can be queried:

SelfSubjectAccessView: Access view for any user, useful for delegating to others.
LocalSubjectAccessReview: Review is restricted to a specific namespace
SelfSubjectRulesReview: A review shows allied actions for a user in a namespace

The use of reconcile allows a check of authorization necessary to create an object from a file. No output means the creation would be allowed.

As mentioned before the serialization for API calls must be JSON, all files in YAML are converted to and from JSON.

The value of resourceVersion is used to determine API updates and implement optimistic concurrency which means an object is not locked from the rime it has been read until the object is written.

The resourceVersion is backed via the modifiedIndex parameter in etc and it\’s unique to the namespace, kind and server. The operations that do not modifiy an object such as WATCH and GET, do not modify this value.

Annotations allow to add metadata to an object, they are key to value maps. Annotations can store more information and in human-readable format, labels are not.

Comparison between Azure Application Gateway V1 and V2

September 10, 2019 iamashishsharmaLeave a comment

Microsoft has announced new version of Azure Application Gateway and its Web Application Firewall module (WAF). In this article, we will discuss about the enhancements and new highlights that are available in the new SKUs i.e. Standard_v2 and WAF_v2.

Enhancements and new features:

Scalability: It allows you to perform scaling of the number of instances on the traffic.

Static VIP: The VIP assigned to the Application Gateway can be static which will not change over its lifecycle.

Header Rewrite: It allows you to add, remove or update HTTP request and response headers on application gateway.

Zone redundancy: It enables application gateway to survive zonal failures which allows increasing the resilience of applications.

Improved Performance: Improvement in performance during the provisioning and during the configuration update activities.

Cost: V2 SKU may work out to be overall cheaper for you relative to V1 SKU. For more information, refer Microsoft pricing page on V2 SKU costs.

The below table shows the comparison between Application gateway V1 and Application Gateway V2:

Features	v1 SKU	v2 SKU
Autoscaling		✓
Zone redundancy		✓
Static VIP		✓
Azure Kubernetes Service (AKS) Ingress controller		✓
Azure Key Vault integration		✓
Rewrite HTTP(S) headers		✓
URL-based routing	✓	✓
Multiple-site hosting	✓	✓
Traffic redirection	✓	✓
Web Application Firewall (WAF)	✓	✓
WAF custom rules		✓
Transport Layer Security (TLS)/Secure Sockets Layer (SSL) termination	✓	✓
End-to-end TLS encryption	✓	✓
Session affinity	✓	✓
Custom error pages	✓	✓
WebSocket support	✓	✓
HTTP/2 support	✓	✓
Connection draining	✓	✓

Microsoft will disable the creation of new Application gateway on V1 SKU soon. So, you may refer this article: https://docs.microsoft.com/en-us/azure/application-gateway/migrate-v1-v2to migrate your existing Application gateway V1 to V2.

Creating Docker Images

September 2, 2019 iamashishsharmaLeave a comment

The distributed systems that can be deployed by k8s are made up primarily of application container images.

Applications = language run time + libraries + source code

Problems occur when you deploy an application to a machine that doesn’t available on the production OS. Such a program, naturally, has trouble executing.
Deployment often involves running scripts which have a set of instructions resulting in a lot of failure cases being missed.
The situation becomes messier if multiple apps deployed to a single machine use different versions of the same shared library.

Containers help solve the problems described above.

Container images

A container image is a binary package that encapsulates all the files
necessary to run an application inside of an OS container.
It bundles the application along with its dependencies (runtime, libraries, config, env variables) into a single artefact under a root filesystem. A container in a runtime instance of the image — what the image becomes in memory when executed. We can obtain container images in two ways:

a pre-existing image for a container registry (a repository of container images to which people can push images which can be pulled by others)
build your own locally

Once an image is present on a computer, it can be run to get an application running inside an OS container.

The Docker Image Format

De facto standard for images
made up of a series of layers of filesystem where each layers adds/removes/modifies files from the previous layer. It’s called an overlay filesystem.

Container images also have a configuration files which specify how to setup the env (networking, namespaces), the entry point for running the container, resource limits and privileges, etc.

The Docker image format = container root file system + config file

Containers can be of two types:

System containers: like VMs, run a fool boot process, have system services like ssh, cron, etc.
Application containers: commonly run only a single app

Building Application Images with Docker

Dockerfiles

It a file that automates the building of container images. Read more here . The book uses a demo app, the source code is available on GitHub. To run the kuar (Kubernetes Up and Running) image, follow these steps:

Ensure you’ve Docker installed and running
Download and clone the kuar repo
Run make build to generate a binary
Create a file named Dockerfile (no extension) containing the following:

FROM alpine

COPY bin/1/amd64/kuard /kuard

ENTRYPOINT [\”/kuard\”]

Next, we build the image using the following command: $ docker build -t kuard-amd64:1 . The -t specifies the name of the tag. Tags are a way to version Docker images. the . tells docker to use the Dockerfile present in the current directory to build the image.

alpine as mentioned in the Dockerfile is a minimal Linux distribution that is used as a base image. More info can be found here.

Image Security

Don’t ever have passwords/secrets in any layer of your container image. Deleting it from one layer will not help if the preceding layers contain sensitive info.

Optimising Image Sizes

If a file is present in a preceding layer, it’ll be present in the image that uses that layer even though it’ll be inaccessible.
Every time a layer is changed, every layer that comes after is also changed. Changing any layer the image uses means that layer and all layers dependent on it need to be rebuilt, re-pushed and re-pulled for the image to work. As a rule of thumb, the layers should be ordered from least likely to change to most likely to change to avoid a lot of pushing and pulling.

Storing Images in a Remote Registry

To easily share container images to promote reuse and make them available on more machines, we use what’s called a registry. It’s a remote location where we can push our images and other people can download them from there. They’re of two types:

Public: anyone can download images
Private: authorisation is needed to download images

The book uses the Google Container Registry whereas I used the Docker Hub. After creating an account on Docker Hub, run the following commands:

$ docker login
$ docker tag kuard-amd64:1 $DockerHubUsername/kuard-amd64:1
$ docker push $DockerHubUsername/kuard-amd64:1 Replace $DockerHubUsername with your username.

The Docker Container Runtime

The default container run time used by Kubernetes is Docker.

Running Containers with Docker

Run the following command to run the container that you pushed to Docker Hub in the previous step: $ docker run -d –name kuard -p 8080:8080 $DockerHubUsername/kuard-amd64:1 Let’s try to unpack everything that command does one thing at a time:

-d tells Docker to run the contain in detached mode. In this mode, the container doesn’t attach its output to your terminal and runs in the background.
—name gives a name to your container. Keep in mind it doesn’t alter the name of your image in anyway.
-p enables port-forwarding. It maps port 8080 on your local machine to your container’s port 8080. Each container gets its own IP address and doesn’t have access to the host network (your machine in this case) by default. Hence, we’ve to explicitly expose the port.

To stop & remove the container, run: $ docker stop kuard $ docker rm kuard

Docker allows controlling how many resources your container can use (memory, swap space, CPU) etc., using various flags that can be passed to the run command.

Cleanup

Images can be removed using the following command: $ docker rmi

Docker IDs can be shortened as long as they remain unique.

Minikube Installation

August 24, 2019 iamashishsharmaLeave a comment

There are lots of other ways to build a kube cluster, such as kubeadm, or my favourite Kubernetes the hard way. However, we will create a kube cluster locally on our workstation using Minikube. I should point that all the things I\’ll demo in this course will work the same way irrespective of how the Kube Cluster was built.

Earlier we talked about how to install minikube, lets now if it has really installed by running a minikube command:

$ minikube version

minikube version: v1.0.1

Similarly you check it\’s status:

$ minikube status

host:

kubelet:

apiserver:

kubectl:

To create a new kubecluster, we run (note this can take several minutes):

$ minikube start

If you open up the virtualbox gui, you should see a new vm called minikube running. If you check the status again, you should now see:

$ minikube status

host: Running

kubelet: Running

apiserver: Running

kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100

Here it says that minikube has also configured kubectl, that\’s done by making changes to kubectl\’s config file. By default that file is located at ~/.kube/config. We\’ll cover more about this file later in the course. But for now we\’ll confirm that this config file is currently configured to point to minikube\’s kube cluster:

$ kubectl cluster-info

Kubernetes master is running at https://192.168.99.100:8443

KubeDNS is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use \’kubectl cluster-info dump\’.

The ip address shown here is the Minikube VM\’s ip address, which should match:

$ minikube ip

192.168.99.100

To check the health of your kub cluster\’s control plane, you can run:

$ kubectl get componentstatus

NAME STATUS MESSAGE ERROR

controller-manager Healthy ok

scheduler Healthy ok

etcd-0 Healthy {\”health\”:\”true\”}

Also to see how many nodes are in our kubecluster, run:

$ kubectl get nodes

NAME STATUS ROLES AGE VERSION

minikube Ready master 4d10h v1.14.1

This command lists out all VMs that has the kubelet component running on it, along with the kubelet\’s VERSION. If you built kubernetes the hardway then the masters won\’t get listed here, since the masters don\’t have the kubelet running on them. We can specify the \’wide\’ output setting to display a little some more info:

$ kubectl get nodes -o wide

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME

minikube Ready master 20h v1.14.0 10.0.2.15 <none> Buildroot 2018.05 4.15.0 docker://18.6.2

Now that we have a kubecluster in place, we can now run the kubectl version command but this time without the –client filter flag:

$ kubectl version –short

Client Version: v1.14.1

Server Version: v1.14.1

By design, to stay lightweight, our minikube based kubecluster is a single node cluster, which acts as both the master and worker node. That\’s fine in a development environment. But in production, you should have multiple node cluster for High Availability, better performance, more CPU+RAM capacity,..etc.

When your not using minikube, you can shut it down:

minikube stop

You can also delete your minikube vm:

minikube delete

The Kubernetes dashboard

You can also monitor your kubecluster via the web browser, bu running:

$ minikube dashboard

This is a really cool tool that let\’s you view and manage your Kubecluster visually. I encourage you to explore this tool as we progress through the course.

MiniKube Installation using Powershell

August 19, 2019 iamashishsharma1 Comment

Minikube is a CLI tool that provisions and manages the lifecycle of single-node Kubernetes clusters running inside Virtual Machines (VM) on your local system. It runs a standalone cluster on a single virtual machine for a quick Kubernetes setup so that you can easily and quickly try your hands at deploying and testing Kubernetes on your own time.

To install & configure Minikube, run the below powershell command & you can have a standalone Kubernetes cluster running locally.

<#
.Synopsis
   Install MiniKube + Kubectl
.DESCRIPTION
   This script downloads the executables for MiniKube, Kubectl, configures Hyper-V as the hypervisor (if not configured already)
   together with configuring a specific network adapter for use with the Minikube virtual machine
.EXAMPLE
   Install-MiniKube

#>

## Check if running as a Administrator (needed for Hyper-V commands)
$currentPrincipal = New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
$currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

## Check HyperV status
$HypervState = (Get-WindowsOptionalFeature -Online -FeatureName:Microsoft-Hyper-V).State

## If missing, enable HyperV
if ($HypervState -eq \"Disabled\")
    {
       $EnableHyperV = Enable-WindowsOptionalFeature -Online -FeatureName:Microsoft-Hyper-V-Management-Powershell,Microsoft-Hyper-V-All -NoRestart
       
       ## If a restart is needed, add registry entry to continue after reboot
       if ($EnableHyperV.RestartNeeded -eq $true)
          {
            ## Set script to re-run after reboot
            Set-ItemProperty -Path \"HKLM:\\Software\\Microsoft\\Windows\\CurrentVersion\\RunOnce\" -Name \"Install-MiniKube\" -Value \"C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Powershell.exe $PSCommandPath\"

            ## And reboot
            Restart-Computer
          }
    }

## Get version number of latest stable release of kubectl
$KubectlVersion = (Invoke-WebRequest -uri https://storage.googleapis.com/kubernetes-release/release/stable.txt -UseBasicParsing).content.Trim() 

## Turn off progress bars to speed up incoming download *sigh*
$ProgressPreference = \"silentlyContinue\"

## Download minikube + kubectl to temp location
$MinikubeUrl = \"https://storage.googleapis.com/minikube/releases/latest/minikube-windows-amd64.exe\"
$MinikubeDl = \"$Env:Temp\\minikube.exe\"
$KubctlUrl = \"https://storage.googleapis.com/kubernetes-release/release/$KubectlVersion/bin/windows/amd64/kubectl.exe\"
$KubctlDl = \"$Env:Temp\\kubectl.exe\"

Invoke-WebRequest -uri $MinikubeUrl -OutFile $MinikubeDl
Invoke-WebRequest -uri $KubctlUrl -OutFile $KubctlDl

## Restore progress bars to default
$ProgressPreference = \"Continue\"

## Create and copy downloads to Minikube directory in Program Files
$MinikubeDst = \"$Env:Programfiles\\Minikube\"

New-Item $MinikubeDst -ItemType Container
Move-Item $MinikubeDl -Destination $MinikubeDst
Move-Item $KubctlDl -Destination $MinikubeDst

## Update PATH environment variable for this session
$env:Path +=\";$MinikubeDst\"

## Update PATH environment variable permentantly
[Environment]::SetEnvironmentVariable(\"Path\", $env:Path + \";$MinikubeDst\", [EnvironmentVariableTarget]::Machine)

## Check for and clear out any previous MiniKube configurations
if (Test-Path -Path \"$HOME\\.minikube\") 
    {
      Remove-Item -Path \"$HOME\\.minikube\" -Force -Recurse
    }

## Get Network Adapter of choice for use with MiniKube
$NetworkAdapter = Get-NetAdapter | Out-GridView -OutputMode Single -Title \'Pick your network adapter to use with MiniKube\' 

## Configure Hyper-V Virtual Switch with Network Adapter chosen previously
New-VMSwitch -Name \"Minikube\" -AllowManagementOS $true -NetAdapterName $NetworkAdapter.Name

## Configure Minikube to use Hyper-V driver and Virtual Network Adapter
minikube config set vm-driver hyperv
minikube config set hyperv-virtual-switch Minikube
minikube config set memory 2048
 
## Start MiniKube
minikube start

Refer : https://dxpetti.com/blog/2019/installing-minikube-with-powershell/

Overview of Kubernetes

August 15, 2019 iamashishsharmaLeave a comment

Kubernetes is an open-source system for automating deployment, scaling and management of containerized applications. source: kubernetes.io
Built from the Google project Borg. Kubernetes is all about decoupled and transient services. Decoupling means that everything has been designed to not require anything else. Transient means that the whole system expects various components to be terminated and replaced. A flexible and scalable environment means to have a framework that does not tie itself from one aspect to the next and expect objects to die and to reconnect to their replacements.
Kubernetes deploy many microservices. Other parties (internal or external to K8s) expect that there are many possible microservices available to respond a request, to die and be replaced.
The communication between components is API call driven. It is stored in JSON but written in YAML. K8s convert from YAML to JSON prior store it in the DB.

Other solutions to Kubernetes are:
• Docker Swarm
• Apache Mesos
• Nomad
• Rancher: Container orchestrator-agnostic system. Support Mesos, Swarm and Kubernetes.

Kubernetes Architecture:

Kubernetes is made of a central manager (master) and some worker nodes, although both can run in a single machine or node. The manager runs an API server (kube-apiserver), a scheduler (kube-scheduler), controllers and a storage system (etcd).
Kubernetes exposes an API which could be accessible with kubectl or your own client. The scheduler sees the requests for running containers coming to the API and find a node to run that container in. Each node runs a kubelet and a proxy (kube-proxy). Kubelet receives requests tu run containers, manage resources and watches over them in the local node. The proxy creates and manage networking rules to expose the container on the network.
A Pod consist of one or more containers which share an IP address, access to storage and namespace. A container in a pod runs an application, and the secondary containers supports such application.
Orchestration is managed though a series of watch-loops, or controllers that check with the API server for a particular object state, modifying the object until declares the desired state.
A Deployment is a controller that ensures that resources are available, and then deploys a ReplicaSet. The ReplicaSet is a controller which deploys and restart containers until the requested number of containers running. The ReplicationController was deprecated and replaced by Deployment.
There are Jobs and CronJobs controllers to handle single or recurring tasks.
Labels are strings part of the object metadata used to manage the Pods, they can be used to check or changing the state of objects without having to know the name or UID. Nodes can have taints to discourage Pod assignment, unless the Pod has a toleration in the metadata.
There are also annotations in metadata which is information used by third-party agents or tools.

Tools:
• Minikube which runs with VirtualBox to have a local Kubernetes cluster
• kubeadm
• kubectl
• Helm
• Kompose: translate Docker Compose files into Kubernetes manifests

Export BACPAC file of SQL database

April 20, 2019 iamashishsharma2 Comments

When you need to create an archive of an Azure SQL database, you can export the database schema and data to a BACPAC file. A BACPAC file can be stored in Azure blob storage or in local storage in an on-premises location and later imported back into Azure SQL Database or into a SQL Server on-premises installation.

Let\’s learn some of the ways to export BACPAC file.

Export BACPAC using Azure Portal

Open your SQL Database and select Export.

Fill the parameters as shown below.

Select your storage account container & enter your SQL Server admin login.

To check the status of your database export. Open your SQL Database server containing the database being exported. Go to Settings and then click Import/Export history

Export BACPAC using SSMS

Right-click the database -> Tasks -> Export Data-tier Application

Save the .bacpac file into local disk.

Export BACPAC using SQLPackage

There is a command line tool that you can also choose to perform the import/export. The latest version is here. Sqlpackage will be installed to “C:\\Program Files\\Microsoft SQL Server\\150\\DAC\\bin” directory.

Example import/export commands

sqlpackage.exe /Action:Export /ssn:[server].database.windows.net /sdn:[database] /su:[user] /sp:[password] /tf:\”C:\\Temp\\database.bacpac\”

sqlpackage.exe /Action:Import /tsn:[server].database.windows.net /tdn:[database] /tu:[user] /tp:[password] /sf:\”C:\\Temp\\database.bacpac\”

The public documentation introduce SQLPackage utility:

https://docs.microsoft.com/en-us/azure/sql-database/sql-database-export#export-to-a-bacpac-file-using-the-sqlpackage-utility

Export BACPAC using Powershell

You can use New-AzSqlDatabaseExport cmdlet to export database bacpac file.

Example:

$exportRequest = New-AzSqlDatabaseExport -ResourceGroupName \"RG01\" -ServerName \"Server01\" -DatabaseName \"Database01\" -StorageKeyType \"StorageAccessKey\" -StorageKey \"StorageKey01\" -StorageUri \"http://account01.blob.core.contoso.net/bacpacs/database01.bacpac\" -AdministratorLogin \"User\" -AdministratorLoginPassword \"secure password\"

Check the status of Database export using the below command:

$exportStatus = Get-AzSqlDatabaseImportExportStatus -OperationStatusLink $exportRequest.OperationStatusLink [Console]::Write(\"Exporting\") while ($exportStatus.Status -eq \"InProgress\") { Start-Sleep -s 10 $exportStatus = Get-AzSqlDatabaseImportExportStatus -OperationStatusLink $exportRequest.OperationStatusLink [Console]::Write(\".\") } [Console]::WriteLine(\"\") $exportStatus

Retrieve Sitecore xDB Cloud 2.0 information using Rest API

March 11, 2019 iamashishsharmaLeave a comment

Sitecore provides REST API references where you can get instant service responses using Xdb API services.

SitecoreRestApi

Refer this for detailed explanation for RestAPI for xDB Cloud Service

1. Get connection strings

This endpoint becomes very useful when you haven’t received any information about xDB Cloud like Connection Strings, ReportingService from Sitecore Support. With this method in place, you will be able to get all the information required for your setup.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/connectionstrings/{licenseId}/{deploymentId}

2. Get Firewall settings V2

This is also one of the important thing to successfully complete the xDB Cloud setup. Even though you have done all configuration right, but your infrastructure not configured properly, it won’t allow application to connect to MongoDB. Hence, no data will be written to xDB Cloud. Use this endpoint to get all URLs and ports specific to URL. And you have to enable the outbound connections over given ports in Firewall.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/firewallsettings/{licenseId}/{deploymentId}?v=2

3. Get history processing status

Once you have triggered the rebuild Reporting database with help of above method and received response as “In Progress”, you will want to know the status of that task to determine weather it got completed successfully or encountered with an error. By triggering this same API as above again, you will get status of that operation in response. Possible values are Done, In Process, Error, and None.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/historyProcessing/{licenseId}/{deploymentId}

4. Get version information

This endpoint is useful to get information about xDB set (DeploymentId) such as Sitecore Version, xDB Cloud Version (1.0/2.0), And Deployment Type (Prod/Non Prod).

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/version/{licenseId}/{deploymentId}

5. Get xDB collection verification dataset

This endpoint also helps like above one to verify weather data is getting stored to xDB Cloud or not by retrieving collections. Collections includes Interactions as well Contacts.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/collection/{licenseId}/{deploymentId}

6. Get xDB consumption

You can utilize this endpoint as a tool to verify whether data is getting stored in xDB Cloud or not. By providing year and month in endpoint URL, you will get informations like Total Interactions, Interactions Added, Total Contacts, Contacts Added etc. per day basis.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/Consumption/{licenseId}/{deploymentId}/{year}/{month}

7. Get xDB set status

This endpoint will be used to ensure xDB Set (DeploymentId) working well or not. This will return Status, Message, and IssueIdOnError. You will find the message if there is any issue with xDB Set. Make a note of IssueIdOnError if you get that in response because you need to provide it to Sitecore Support team when you contact them about the issue with your xDB Set.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/{licenseId}/{deploymentId}/status

8. List xDB sets

Once you have obtained the token, first thing you wish want to get is list of xDB sets. In response you will get all xDB set listed and DeploymentId for each. Once you have noted LicenseId and DeploymentId, you will be able to make API calls to get information specific to xDB set (DeploymentId). This endpoint becomes very handy when you haven’t received DeploymentId(s) from Sitecore Support.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/{licenseId}

9. SSO Encode Sitecore license

You must use a valid authentication token whenever you make a call to the xDB Cloud API. Which can be generated by using a valid Sitecore license file to call the SSO Encode Sitecore License endpoint. You must include the generated token as a HTTP header in all other requests called X-ScS-Nexus-Auth

https://gateway-sso-scs.cloud.sitecore.net/api/License/Encode/

10. Trigger history processing (rebuild the Reporting database)

This is a very useful endpoint which will be used to rebuild the Reporting database. Make note that you won’t be able to rebuild the reporting database using admin page in case of xDB Cloud. This is the method which will do the work for you. When calling this method, it should return “In Process” as response. That means rebuild of reporting database for that specific xDB Set has been started.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/historyProcessing/{licenseId}/{deploymentId}

11. Cancel history processing

This endpoint is used to cancel rebuild the Reporting database. You should have licenseID and deploymentID to call this endpoint.

https://gateway-xdb-scs.cloud.sitecore.net/api/xdb/cancel/historyProcessing/{licenseId}/{deploymentId}

Sitecore xDB Troubleshooting

January 7, 2019 iamashishsharmaLeave a comment

Below are the some of the useful xDB troubleshooting reference:

1. Determine the interactions for each facet by month, run against your xDB shards:

select facetKey,
DATEPART(year,lastmodified) as ‘Year’,
DATEPART(month,lastmodified) as ‘Month’,
count(distinct interactionId) as ‘NumberOfInteractions’
from [xdb_collection].[InteractionFacets]
group by facetKey, DATEPART(year,lastmodified), DATEPART(month,lastmodified)
order by facetKey, DATEPART(year,lastmodified), DATEPART(month,lastmodified)

2. Determine the number of visits by language and month, run against your reporting database:

select SiteNameId,
DimensionKey,
DATEPART(year,[Date]) as ‘Year’,
DATEPART(month,[Date]) as ‘Month’,
sum(visits) as ‘NumberOfInteractions’
from [dbo].[Fact_LanguageMetrics] lm
inner join [dbo].[DimensionKeys] dk on lm.DimensionKeyId = dk.DimensionKeyId
group by SiteNameId,
DimensionKey,
DATEPART(year,[Date]),
DATEPART(month,[Date])
order by SiteNameId,
DimensionKey,
DATEPART(year,[Date]),
DATEPART(month,[Date])

3. Extract Email addresses from xDB shards:

select c.contactid, c.created, c.LastModified, cf.FacetData, JSON_VALUE(cf.FacetData, ‘$.PreferredEmail.SmtpAddress’) AS Email
from [db-shard0db].[xdb_collection].[ContactFacets] cf
inner join [db-shard0db].[xdb_collection].[Contacts] c on c.ContactId = cf.ContactId
where cf.FacetKey = ‘Emails’

4. Determine xDB data flow and pinpoint/analyze issues:

To check what user you are within xDB you will need to have the value of your SC_ANALYTICS_GLOBAL_COOKIE. Copy the value without ‘| False’.

To force a Session_End you can upload the following .aspx file to a location on your website: “here– Now browse through your website(s), and end the session by hitting the sessionkill.aspx. This should trigger the the xDB data to be send towards your SQL shards.

Connect to your SQL server and execute the query below on every shard:

SELECT *
FROM [xdb_collection].[DeviceProfiles]
where DeviceProfileId like ‘%VALUEFROMCOOKIE%‘
order by LastModified desc

The ID that is found in column “LastKnownContactId” – will map a new “ContactId” – “XXX” – This ID will be used for identification, for example in the Interaction table.

To check whether page events are written to the “interaction” tables

SELECT * from

SELECT * from [xdb_collection] .[Interactions]
where ContactId = ‘XXX’

In case you are missing interactions, you might want to check Application Insights whether there are Session_End submits that are failing [KQL] :

traces
| where timestamp > ago(30d)
| where operation_Name contains “GET /SESSION END”

For more detailed information related to xDB troubleshooting refer the below document:

Troubleshooting xDB data issues

Troubleshooting xDB Cloud

Authenticating using a Service Principal

November 2, 2018 iamashishsharmaLeave a comment

Azure service principal is a security identity created within Azure Active Directory & it is used by user-created apps, services, and automation tools to access specific Azure resources. You can assign permissions to the service principal that are different than your own Azure account permissions.

Creating a Service Principal

Create an Application in Azure Active Directory which will create an associated Service Principal.

Go to Application & you will need to take note of the \”Application ID (client_id)\” and the \”Directory (tenant) ID(tenant_id) . Then, click on Certificates & Secrets from left navigation bar.

Click on New client secret then fill the description & select expiry year.

Once the Client Secret has been generated it will be displayed on screen.This is only displayed once so be sure to copy it now (otherwise you will need to regenerate a new secret). This value is the client_secret you will need.

Assign Role to Service Principal

Once Service Principal will be created in the Azure AD, we can grant it permissions in Subscription or in Resource Group level.

Go to Subscription you wish to use , then click Access Control (IAM), and then Add > Add role assignment. For more details related to IAM role refer this: Azure built-in roles. If you want to assign role in Resource Group level, then go to Resource Group > Access Control (IAM) > Add > Add role assignment.

Powershell script to Login via Service Principal

Fill the parameters & run the below command to login as a Service Principal:

$client_id =\"<>\"
$tenant_id= \"<<Directory ID"
$client_secret=\"<>\"
$secret = ConvertTo-SecureString $client_secret -AsPlainText -Force
$Cred = New-Object System.Management.Automation.PSCredential($client_id, $secret)
Add-AzureRmAccount -Credential $Cred -TenantId $tenant_id -ServicePrincipal

Share this:

Share this:

Share this:

Share this:

How to enable Change Analysis Feature

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

Enhancements and new features:

Share this:

Share this:

Share this:

Refer : https://dxpetti.com/blog/2019/installing-minikube-with-powershell/

Share this:

Share this:

Export BACPAC using Azure Portal

Export BACPAC using SSMS

Export BACPAC using SQLPackage

Export BACPAC using Powershell

Share this:

Share this:

Share this:

Share this: