Microsoft Restricted Free Usage of Azure DevOps Pipeline

 Microsoft has changed the policy to allow the free tier of a hosted agent pools for public and private projects of newly created DevOps organizations by citing the reason that many are abusing this feature by sending a huge amount of traffic on these hosted agents pools. 

Due to this reason, many are getting the following error during the build pipeline.

No hosted parallelism has been purchased or granted. To request a free parallelism grant, please reach out to azpipelines-freetier@microsoft.com with your name and organization name.

Microsoft comes with the some temporary alternative approach until the automate is implemented to grant the permission for those users who requires the free hosted agent pools. 

Now users needs to be drop the email to get the free tier access based on the project visibility types which is explained below

Private Project

To get the free tier access of Microsoft hosted agent pools for private projects, send the email to azpipelines-freetier@microsoft.com with the following details.

• Your Name
• Microsoft azure DevOps organization name (dev.azure.com/yourorganization)

Public Project

To get the free tier access of Microsoft hosted agent pools for public projects, send the email to azpipelines-ossgrant@microsoft.com with the following details.

• Your Name
• Microsoft azure DevOps organization name (dev.azure.com/yourorganization)
• Link of the repositories you want to build
• Short description of your project.

Youtube Video: https://www.youtube.com/watch?v=WiHxt5YnWYs

What is Azure Devops?

 Azure DevOps is also known as Microsoft visual studio team services (VSTS) is a set of collaborative development tools built for the cloud. 

VSTS was commonly used as a standalone term, and Azure DevOps is a platform which is made up of a few different products, such as:

• Azure Test Plans
• Azure Boards
• Azure Repos
• Azure Pipeline
• Azure Artifacts

The azure pipeline is the CI component of azure DevOps. 

An azure pipeline can connect to any number of source code repositories such as Azure Repos, GitHub, Tests, to grab code and artifacts for application delivery.

Azure Boards:

Azure board is basically used for planning, tracking for Work Items, Backlog, and Sprints. Azure Board is also called an ALM (application life cycle management) tool just like other ALM tools JIRA, SpiraTeam, etc. Inside the Azure board, we can create workflows, issue types, epic, and many other components of ALM.

Azure Repos:
Azure Repos is a place where we can create, manage and store versions of our codebase. We can use it for code review and other version control functionality.

Azure Repos basically provides us two types of version control systems now GIT (Distributed version control) and TFVC (centralized version control).

Azure pipeline:

Azure pipeline is one of the automation processes of DevOps which allows us to create a build, release, test, and deploy on the target machine. We can say in simple words Azure pipeline is the combination CI/CD.

Azure Artifacts:

Azure Artifacts is one of the extensions of Azure DevOps which helps us to create, host, manage and share packages across the team. Azure Artifacts supports multiple types of packages e.g NPM, Nuget, Maven, Python, etc.. Azure Artifacts are basically a collection/ output of dll, rpm, jar and many other types of files. One of those extension based files may contain metadata.

Azure test Plans:

Azure test plans support advance test management solutions for customer feedback, UAT, Automation testing, and manual testing.

Tasks That Can be performed on Azure Devops:

What can Azure DevOps do?

• Planning
  As discussed above, the Azure board is an ALM tool. So Azure board helps us to maintain an Agile process which means Azure DevOps helps us to plan, track development history for individual developers and detects issues using a kanban and scrum. 
  
  
• Develop
  To manage our workspace and code repository multiple types of version control are already integrated with Azure DevOps. Almost all features related DevOps already inbuilt inside.
  
  
• Integration/Build
  We can use continuous integration and continuous deployment (CI/CD) with the help of the Azure pipeline. There are multiple types of job agents, and templates are already available inside. 
  
  
• Package
  As we know we have already an inbuilt extension available, Azure Artifacts, so we don't need to go anywhere for any type of packages.
  
  
• Testing
  As we know Testing is an important part of the DevOps process. So Azure DevOps provides multiple types of inbuilt testing templates and management tools.
  
  
• Release
  As we know the software release process is a very critical moment of DevOps. So Azure DevOps has already multiple types of inbuilt features to make the process easy and risk-free.
  
  
• Configure/Operate
  As per my experience, it's very easy to configure and operate, because Azure DevOps has a simple process to configure organization, kanban, sprints, etc..
  
  
• Monitor
  Each phase of the DevOps life cycle should be monitor because it helps to ensure the health, performance, and reliability of your application.
  
  
• Reporting
  Azure DevOps has a dashboard where we can see reports, a summary and can generate an analytical report with BI integration. 
  
  
• Wiki
  Azure DevOps Wiki helps us to distribute information, share knowledge and collaborate across team and stakeholder. A Wiki can be used to explain about the project's user stories, sprints, release notes, etc..

Link: https://www.youtube.com/watch?v=WiHxt5YnWYs

Dockerfile

A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image

The Docker build command executes the Dockerfile and builds a Docker image from it.

 A Docker image typically consists of:

• A base Docker image on top of which to build your own Docker image.
• A set of tools and applications to be installed in the Docker image.
• A set of files to be copied into the Docker image (e.g configuration files).
• Possibly a network (TCP / UDP) port (or more) to be opened for traffic in the firewall..etc

Dockerfile Structure

A Dockerfile consists of a set of instructions. Each instruction consists of a command followed by arguments to that command, similar to command line executables. 

Here is a simple example of a Dockerfile:

# The base image

FROM ubuntu:latest

# More instructions here that install software and copy files into the image.

COPY    /myapp/target/myapp.jar    /myapp/sriniapp.jar

# The command executed when running a Docker container based on this image.

CMD echo Starting Docker Container

Refer: https://www.youtube.com/watch?v=QLENvSMSD-0

5G

 5G is the 5th generation mobile network.

It is a new global wireless standard after 1G, 2G, 3G, and 4G networks. 5G enables a new kind of network that is designed to connect virtually everyone and everything together including machines, objects, and devices

Where 3G and 4G technologies have enabled us to browse the internet, use data driven services, increased bandwidths for streaming on Spotify or YouTube and so much more, 5G services are expected to revolutionize our lives. by enabling services that rely on advanced technologies like AR and VR, alongside cloud based gaming services like Google Stadia, NVidia GeForce Now and much more.

It is expected to be used in HD cameras that help improve safety and traffic management, smart grid control and smart retail too.

Previous generations:

First generation - 1G

1980s: 1G delivered analog voice.

Second generation - 2G

Early 1990s: 2G introduced digital voice (e.g. CDMA- Code Division Multiple Access).

Third generation - 3G

Early 2000s: 3G brought mobile data (e.g. CDMA2000).

Fourth generation - 4G LTE

2010s: 4G LTE ushered in the era of mobile broadband.

5G will help edge computing grow:

The 5G network’s speed should be 10 times faster than what the 4G network allows. That improvement opens possibilities for far-away sensors to instantly give updates about the connected devices. 

That increase in real-time processing will require new equipment with enhanced capabilities.

It’ll also raise the demand for edge computing since that method processes data at the borders of a network rather than transferring the data a significant distance first

To know more about Edge computing refer below link:

https://nextsrini.blogspot.com/2020/05/edge-computing.html

JIRA Rest API using Python

The Jira REST API enables you to interact with Jira programmatically. Use this API 

to build apps, script interactions with Jira, or develop any other type of 

integration.

The URIs for resources have the following structure:

https:///rest/api/3/

There are so many libraries like JIRA,request to access the JIRA services.
  

In the below code snippet, i had used request library and to save the results used 

Pandas library.

Pandas in the best library to save the results to Excel workbook.

Pandas library can be installed using either PIP/PIP3/easy-install. Make sure you 

are having right SSL certificate's before install pandas in intranet.

pip install pandas  
  
Sample Code Snippet:
  
import requests
from requests.auth import HTTPBasicAuth
import json
import pandas as pd

pd.set_option('display.max_columns', 500)
# url = "https://ABCD.atlassian.net/rest/api/3/issuetype"
url = "https://ABCD.atlassian.net/rest/api/3/search"

auth = HTTPBasicAuth("xxxxxxx", "yyyyyyyyyyyyyy")

headers = {
    "Accept": "application/json"
}
query = {
    'jql': 'project = ABCD'
}

response = requests.request(
    "GET",
    url,
    headers=headers,
    params=query,
    auth=auth
)
resdata = json.dumps(json.loads(response.text))
# print(json.dumps(json.loads(response.text), sort_keys=True, indent=4, 

separators=(",", ": ")))

if response.status_code != 200:
    print('Status:', response.status_code, 'Headers:', response.headers,
          'Error Response:', response.json())
    exit()
else:
    print('Server Connected.Please wait.....')

#for fetching all sprints data
df = json.loads(resdata)
maxresults = df['maxResult']
print(df['total'])
writer = pd.ExcelWriter('Jira_issues.xlsx', engine='xlsxwriter')
dk.to_excel(writer, sheet_name='Jira_Issues', index=False)
writer.save()

Blockchain Versions

Blockchain technology has gained popularity because it offers decentralized and transparent storage to record transactions and data.

Blockchain Version 1.0: Cryptocurrency:

Blockchain 1.0 is the first cryptocurrency, which records the bitcoin transactions on the public ledger in a secure method. The implementation of Distributed ledger technology was the reason behind its first version.

It is a permission-less blockchain where any participant can perform a valid transaction of bitcoins.

It works as a simple state machine, which can be used as a platform for the bitcoin transaction. State machine manages the transition between valid states similarly Blockchain records and manages the bitcoin transactions between valid participants of the network.

Blockchain Version 2.0: Smart Contract:

The new concepts in the blockchain are Smart Contracts,  it is a small self-executing computer :programs that exist in the blockchain. In Blockchain 2.0, Bitcoin is replaced by ethereum. 

The smart contract is one of the most successful blockchain applications, it helps in reducing transactions cost-effectively. Ethereum network is the best platform for creating and executing smart contracts. This is used as a replacement for the traditional contracts.

Blockchain Version 3.0: Dapps:

The Dapp is a short form for Decentralized application. Decentralized applications run  opposite to centralized applications running on a single computer. 

A Dapp is like a conventional app, it can have frontend (written in any language) that can make calls to its backend. A Dapp can host its frontend on decentralized storages, including Ethereums Swarm.

Decentralized apps do not need to run on top of a network of blockchains. There are many decentralized applications running on a P2P network, like BitMessage, BitTorrent, Tor, Popcorn, etc.

Vulnerabilities in Python Applications

Below are the types of security vulnerabilities that Python developers should be concerned with:

SQL INJECTIONS (SQLI)

A malicious user controls the execution of SQL statements for an application at the backend database server. 

There are four sub-classes in SQLi:

• In-band SQL Injection / Classic SQLi
• Inferential / Blind SQL injection
• DBMS SQLi
• Compounded SQLi (Eg: Strom Worn)
• SQLi with inadequate authentication
• SQLi with DDoS attacks
• SQLi with DNS hijacking
• SQLi with XSS

CROSS SITE SCRIPTING (XSS)

In XSS, a malevolent user can trick any web application to steal stored cookies, saved passwords, and script code that served unsuspecting users of that application.

CROSS SITE REQUEST FORGERY (CSRF)

This security vulnerability occurs when a compromised website is forced to perform an action by another logged-in user like clicking on a button. Also, it includes the hacking or logging into of a website with others’ login credentials.

LDAP (LIGHTWEIGHT DIRECTORY ACCESS PROTOCOL) INJECTIONS

This vulnerability occurs when a malicious user inserts/modifies LDAP statements that lead to speculations.

COMMAND INJECTIONS

Here, a malicious user executes OS commands on a web server by abusing it in order to insert their own commands to gain complete control over the server.

XPATHI

This occurs when a malevolent user intentionally passes data to a website. They can use that interaction to find out how the data is structured in XML, or they can access secured data that they can’t access normally.

Security Scanners

Python Taint (PYT) – Static Analysis Tool: This utility is used for identifying command injection, XSS, SQLi, interprocedural, path traversal HTTP attacks in Python web apps. Python Taint is based on the Control flow graphs, data flow analysis and fixed points that are theoretical foundations built using the Flask framework.

Tinfoil Security Website Scanner: Tinfoil is an affordable security scanner for Python & Django that helps find holes in web servers and applications and also tells you the ways to fix them.

Bandit – AST Based Static Analyzer: It’s an OpenStack security linter that identifies the common security risks in Python programming. It is distributed using pip. To install bandit from source, we can use the command python setup.py install after downloading the pypi source tarball. You can even access the reports from bandit.

Pyntch – Static Code Analyzer: Pyntch (PYthoN Type CHecker) helps in detecting runtime errors such as exceptions, not found attributes and variable type missmatchings. It supports Python 2.x currently. It won’t address style issues like Pychecker or Pyflakes, but it works pretty fast and efficient in scanning thousands of lines within a minute.

Spaghetti Security Scanner: Spaghetti an open-source web application security scanner built on Python version 2.7. It detects default files, misconfigurations, and insecure files, and it supports numerous frameworks including Django, CherryPy, CakePHP, and others. 

The tool is capable of finding attacks like admin panel, cookie security, credit card/email/private IP disclosures, SQL injections, ShellShock, Struts-shock, Apache ModStatus, Anonymous cipher, and others.

Rough Auditing Tools for Security (RATS): It’s a free tool that scans languages like C, C++, PHP, Perl and Python and emphasizes the errors that are related to security like TOC (Time of Check), TOU (Time of Use), Buffer overflows and Acunetix. Manual code introspection is still important, but this tool still greatly assists us.

PyDbgEng – Windows Debugging Engine’s Python Wrapper: It helps in debugging user mode, kernel mode, software and hardware breakpoints, etc. With the help of this PyDbgEbg, you can do fault injection, fuzzing of applications, and unpacking executables automatically.

Python-ptrace: It’s an opensource debugging tool that uses ptrace developed and written in Python. Here, ptrace works as an tracer that hands the system calls in Linux, BSD and Darwin.

vdb / vtrace – Debugger for Exploit Malware Analysis: Here, VDB refers to a dynamic debugging element; vtrace refers to a platform that’s used in debugging frameworks implemented in Python. Vdb utilizes vtrace.

Immunity Debugger – Python Penetration Testing Tool: It uses python scripts and supports Windows with Graphical user interface and command line debuggers.