AWS Cloud AWS CLOUD by Editorial Staff March 4, 2023 written by Editorial Staff AWS Solution Architect Associate Exam Preparation – Notes Because standard queues are designed to be massively scalable using a highly distributed architecture, receiving messages in the exact order they are sent is not guaranteed Standard queues provide at-least-once delivery, which means that each message is delivered at least once. FIFO queues provide exactly once processing, which means that each message is delivered once and remains available until a consumer processes it and deletes it. Duplicates are not introduced into the queue. All messages have a global unique ID that Amazon SOS returns when the message is delivered to the message queue. The ID isn’t required to perform any further actions on the message, but it is useful for tracking the receipt of a particular message in the message queue. When you receive a message from the message queue, the response includes a receipt handle that you must provide when deleting the message. An Amazon SQS message can contain up to 10 metadata attributes. You can use message attributes to separate the body of a message from the metadata that describes it. This helps process and store information with greater speed and efficiency because your applications don’t have to inspect an entire message before understanding how to process it. Amazon SQS long polling is a way to retrieve messages from your Amazon SQS queues. While the regular short polling returns immediately, even if the message queue being polled is empty, long polling doesn’t return a response until a message arrives in the message queue, or the long poll times out. In general, you should use maximum 20 seconds for a long poll timeout. Because higher long-poll timeout values reduce the number of empty ReceiveMessageResponse instances returned, try to set your long-poll timeout as high as possible. Long polling timeout can be between 0 sec and 20 sec (maximum). When Amazon SQS returns a message to you, the message stays in the message queue whether or not you actually receive the message. You’re responsible for deleting the message and the deletion request acknowledges that you’re done processing the message. . if you don’t delete the message, Amazon SOS will deliver it again on when it receives another receive request. When you issue a DeleteMessage request on a previously deleted message, Amazon SQ5 returns a success response. FIFO queues never introduce duplicate messages. Both standard and FIFO queues support SSE AWS SQS is both PCI DSS level 1 certified and HIPAA compliant. There is a 120,000 limit for the number of in flight messages for a standard queue and 20,000 for a FIFO queue. Messages are in-flight after they have been received from the queue by a consuming component, but have not yet been deleted from the queue. Each Amazon SOS message queue is independent thin each region and can not be shared across regions. When you create a security group, it has no inbound rules. Therefore, no inbound traffic originating from another host to your instance is allowed until you add inbound rules to the security group. By default, a security group includes an outbound rule that allows all outbound traffic. If your security group has no outbound rules, no outbound traffic originating from your instance is allowed. Security groups are stateful-if you send a request from your instance, the response traffic for that request is allowed to flow in regardless of inbound security group rules. Responses to allowed inbound traffic are allowed to flow out, regardless of outbound rules. ICMP traffic is always regardless of the rules (ex. even if 0.0.0.0/0 is specified for ICMP). . Incoming traffic is allowed based on the private IP addresses of the instances that are associated with the source security group (and not the public IP or Elastic IP addresses). . If you specify a single IPv4 address, specify the address using the /32 prefix length. If you specify a single IPv6 address, specify it using the /128 prefix length. The rules you create for use with a security group for a VPC can’t reference a security group for EC2-Classic, and vice versa IAM/Security. March 4, 2023 0 comment 0 FacebookTwitterPinterestEmail
Kubernetes Kubernetes Architecture by Editorial Staff March 4, 2023 written by Editorial Staff What is Kubernetes: KBS is an Open-Source container orchestration system. Automated application deployment, scaling and management. (We won’t be informed if the container goes down in docker). Why to use K8S: Can’t live with single server deployment Inter host communication of containers Deploying and updating software at scale Auto-Healing Logging and Monitoring. (in docker we are stuck to a single system) Kubernetes Architecture: Containers are great at accomplishing things like: Software Portability-Running software consistently on different machines. Isolation-Keeping individual pieces of software separate from one another. Scaling-Increasing or decreasing resources allocated to software as needed. Automation-Automating processes to save time and money. Efficient Resource Usage-Containers use resources efficiently, which saves money Advantages of Containers: These are a few of the main advantages of containers: The isolation and portability of VMs. More lightweight than VM-Less resource usage. Faster than VM-Containers can start up in second not minute. To create a container docker needs an Image Since they don’t have full blown OS: Cpu and other resources are allocated to application rather than os This reduces the overhead of OS patching Savings on Cost as containers don’t have OS’s (license cost) Why Containers ? Containers are a powerful tool! They have had a tremendous impact on how the industry does IT. They can help you to Move Faster Automate More Increase Reliability For Less Money What are containers used for ? Image is a package or a template just like a VM template.it is used to create one or more containers. Containers are running instances of images that are isolated and have their own environments and set processes. March 4, 2023 0 comment 0 FacebookTwitterPinterestEmail
Kubernetes Kubernetes Architecture by Editorial Staff March 4, 2023 written by Editorial Staff So what are Containers ? Before going to containers, lets understand VM’s and history: create the new image Docker Commit: Creates an image from a container. Docker Export: Creates a tar file from a container. Docker import: Creates a image from a tar file. Docker Network: Till now we have seen docker is getting ip addresses right, from where is it getting ip address?? Docker’s networking subsystem is pluggable. ie., it is like a plugin we will not be implementing as part of the Containerd Shrim Runc Dockerd: Listem for Docker Engine API Requests Containerd: Introduced in Docker 1.11 Responsibility of managing container life cycle Containerd is the executor for containers Executing containers by calling RUNC with the right parameters to run containers. Runc: Container used runc to do all the linux work by interacting with the kernel By interacting with the kernel, it creates the container process and another process called as containerd-shim Once the container starts, the runc get exists Containerd-shim: It allows you to run daemon less containers When the container process dies, containerd-shim will inform the status to containerd and exists Docker Images: A Docker Image is a file, used to execute code in a container. It’s a Read only Template. The containers are not so self-sufficient to host the application, and to work on a highly scalable application, so that’s the reason people started looking toward an options like orchestration tool (Ex: Kubernetes) March 4, 2023 0 comment 0 FacebookTwitterPinterestEmail
Kubernetes Kubernetes Architecture by Editorial Staff March 4, 2023 written by Editorial Staff Kubernetes Cluster Creation: There are multiple ways to create a cluster. Following are the few ways to do that: Minikube Kubeadm Kops Installing through Binaries Managed Kubernetes cluster(Ex GXE, AKS, EKS) We will be going through creating a X85 Cluster using Kubeadm Smaller than VMs-Container images can be measured in megabytes, not gigabytes. . All of these add up to faster and simpler automation! Limitations of Containers: These are some limitations of containers Less flexibility than VMS-You can’t run a Windows container on a Linux machine (yet). Introduces new challenges around this area. Docker Introduction: Docker is a platform for developers and sys admins to develop ship and run applications by using containers. This means that Docker is a piece of software that is designed to implement and support containers. . Docker is primarily a container runtime. At its core, Docker allows you to run containers on systems. It also offers a variety of tools for creating and managing containers and container images. It is currently the industry leader in container runtimes. Other CRT’s are rkt, container, pod man. Container Vs Image: Image is a package or a template just like a VM template it is used to create one or more containers. To enable SSH password authentication, you must SSH in as root to edit this file: /etc/ssh/sshd_config PasswordAuthentication yes sudo service ssh restart Note: if we don’t want to give the user the full sudo access, rather want to provide him only access to docker execute the below command. sudo usermod-aG docker $USER Docker Architecture: Dockerd and Docker Engine are synonyms, both are the same. Docker uses Client-Server architecture. We will be looking at the Docker architecture and core components of a docker engine. Docker was first released to the public in 2013. The Docker Engine consists of Docker Daemon, the REST API and Docker CLI. The Docker Daemon, is the actual server or process that is responsible for creating and managing objects, such as images, containers, volumes, and networks on a host. The Docker CLI, is the command line interface that we run commands to manage objects in Docker. The REST API provides an interface to manage objects in Docker. How does Docker manage containers on a host? So what are Containers ?: Before going to containers, lets understand VM’s and the history: Containers are all about portable software. They are a technology that allows you to run software on a variety of systems, including a developer’s laptop, all the way to a production system. This speeds up deployment, simplifies automation and ensures your code can run consistently in production, as well as everywhere else! Like virtual machines, containers wrap your software in a standardised environment that allows it to run consistently on varied machines. But containers are smaller, use fewer resources, and are easier to automate than virtual machines. Containers are completely isolated environments that can have their own services, processes network interfaces their own mounts just like VM’s, except they all share the same OS kernel. The major difference b/w vm and container is Um’s have a full blown OS, whereas containers are isolated areas don’t have full blown OS). Since containers are an isolated area. what’s happening on one container can reflect on another container. Each container have there own below components: CPU RAM Storage Network (ip address) Images are built using the docker image build command The build is run by docker daemon. The docker daemon runs the instructions in the Dockerfile one-by-one, committing the result of each instruction to a new image if necessary, before outputting the ID of your new image. e-Dockerfile Instructions: FROM: Inimalises a new build stage and sets the Base Image RUN: Will execute any commands in a new layer CMD: Provides a default for an executing container. There can only be one CMD instruction in a Dockerfile LABEL: Adds metadata to an image EXPOSE: Informs Docker that the container listens on the specified network ports at runtime ENV: Sets the environment vanable <key> to the value <value> ADD: Copies new files, directories, Untar the tar files directories or remote file URLs from care and adds them to the hiesystem of the image at the path <dest> COPY: Copies new files or directories from <src> and adds them to the file system of the container at the path <dest> March 4, 2023 0 comment 0 FacebookTwitterPinterestEmail
Kubernetes Kubernetes Architecture by Editorial Staff March 4, 2023 written by Editorial Staff Kubernetes Objects: Namespaces: Namespaces is logically dividing our cluster to deploy resources. Namespaces provide a way to keep your objects organized within the cluster. Every object belongs to a namespace. Namespaces are a way to separate and organize objects in a cluster. When no namespace is specified, the cluster will assume the default namespace Default Namespace: For objects with no namespace mentioned Kube-system: For objects created by the kubernetes system. Each Kubernetes resource can only be in one namespace. Working with Kubectl: What is a Kubectl ? Kubectl is a command line tool that allows you to interact with Kubernetes. Kubectl uses the Kubernetes API to communicate with the cluster. Imperative. Declarative The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. Monolithic Applications: Few years back we used to see the majority of the apps getting 404 errors, because if some components are not working, everything also breaks. Because it’s behaving like a single unit, a complete monolithic application. When we say it’s a monolithic application, it’s a bulky application, in which the frontend, backend middleware and everything will be present in a single/complete unit. If we want to deploy, we need to deploy the whole component, we can’t deploy individual components. Micro services Overview: Micro services are a method, which we design. Micro services is a way of architecting, and micro services will be running in containers, and those containers are created by docker, and those containers are orchestrated/managed by a orchestration tool called Kubernetes. Split the application into a set of smaller, interconnected services. ETCD: ETCD is a backend data store for the Kubernetes cluster. . ETCD is a distributed, reliable key-value store that is Simple, Secure and Fast. It provides high availability storage for all data relating to the state of the cluster. Consistent and highly-available key value store used as Kubernetes’ backing store for all cluster data. The etc datastore, stores information regarding the cluster such as nodes, pods, config, secrets, accounts, roles, binding etc. Every information we see when we execute kubectl get command is from etcd server. Every change you make to the cluster, such as adding additional nodes, deploying pods, or replicas sets are updated in the etcd server. Only once it is updated in the etcd server the changes are considered to be completed. Kube controller manager: March 4, 2023 0 comment 0 FacebookTwitterPinterestEmail
Kubernetes Kubernetes Architecture by Editorial Staff March 4, 2023 written by Editorial Staff Kubernetes Introduction: Till now we have gone through containers and learned about the docker containers, and from this section we will be starting our journey towards K8S. . Firstly we will understand the drawbacks of a containerized platform………. Containers are having many advantages compared to vm and on prim, that’s the main reason many apps A Docker image is a file, used to execute code in a container. It’s a Read only Template. Docker image is created using a Dockerfile. A Docker image is made up of Multiple Layers. Every Docker Image starts with a Base image. The docker run command creates a container from an image. The docker commit command creates an image from a container. A docker image consists of layer built on top of each other. Docker uses Union File System (UFS) to build an image. Image is shared across containers. Container Image + Writable top layer. In docker the version of your application image is referred to as a tag. Each layer represents an instruction in the image’s Dockerfile. Each layer, except the very last one, is read-only. Each layer is only a set of differences from the layer before it Containers add a new writable layer on top of the underlying layers. All changes made to a running container are made to the Container layer. The writable layer is deleted when the container is deleted. The image remains unchanged. Basic docker Image creation process is choose a base image add your installation/configuration steps Every container has non-persistent data and this storage gets created with the container, which is the read write layer. l.e., when the container is deleted the data is also inside it gets deleted as its non-persistent. Persistent data is the data which we want to store, and this is not ephemeral. We want to stick around, this can be achieved by using docker volumes. Volume data lives outside the life of the container. Going back to our example, if our application is talking to a database, we want to make sure our data is persistent and therefore live in a volume. Non persistent data is commonly referred to as local storage. Linux system storage can be found at /var/lib/docker/<name_of_storage> We will be discussion about 3 types of Volumes: Bind Volume TempFS Docker Volumes Docker Bind Mount: March 4, 2023 0 comment 0 FacebookTwitterPinterestEmail
Kubernetes Kubernetes Architecture by Editorial Staff March 4, 2023 written by Editorial Staff Cluster Architecture: The purpose of kas is to host the applications in the form of containers in an automated fashion, so that we can easily deploy as many containers as are required. There are many components that will make this happen. Master node is also known as the controles control plane node. Master node: It is mainly used for managing, planning, scheduling and monitoring the nodes and does all of these using Control plane components. Worker node: Worker nodes are the one that is hosting your application. Worker node is also known as data plane. Kube-apiserver: Kubernetes API. The API server is the front end for the Kubernetes control plane. The kube api-server is responsible for orchestrating all operations within the cluster. When we run the kubeckicommand, the kubectl utility is in fact reaching the api server server. It exposes the Kubernetes API which is used by external users to perform management operations on clusters as well as various controllers to monitor the state of the cluster and make the necessary changes as required and by the worker node to communicate with the server. The kube api-server first authenticates the requests and validates. It then retrieves the data from the etcd server and responds back with the required information system. Bridge: default network driver Bridget is the default network that is attached to a container. The bridge network is an internal private network created by docker on the host. All containers attached to this network by default and they get an internal IP address. Usually in the range 172.17.xx. The containers can access each other using the Internal IP if required. To access any of these containers from the outside world we need to use port forwarding. Another way to access the containers externally is to attach to the host network. This takes out any isolation b/w docker host and the docker container. With the none Network, containers don’t have any access to the external network or other containers. They run in an isolated network. Docker Storage: In this section, we shall see how data is stored in Docker. By default all the files we write will be present in the Writable layer. The data doesn’t persist when that container no longer exists. Non persistent data is going to be ephemerali.e.. If this data is going to be deleted there is nothing we can lose. Good example is application code, because it is going to be tied to the life cycle of the container. By using docker image as build artifact we are baking our code into the image. And anytime we go and Bind mounts have been around since the early days of Docker. They have limited functionality compared to volumes. With bind mount, a file or directory on the host machine is mounted into a container There are 2 ways of creating bind Mounts: Mount flag: Volume flag A file or directory on the host machine is mounted into a container. File or Directory does not need to exists on the Docker host already. It is created on demand if it does not yet exists with -v option. Docker tmpfs: A tmpf mount is not persisted on disk. It can be used by a container during the life cycle of the container. It is mainly used to store non-persistent state or sensitive information. Can’t share tmpfs mounts between containers. Docker tmpfs: Preferred mechanism for persisting data generated by and used by docker containers. Dockerfile: What is the Dockerfile? Dockerfiles are instructions. They contain all of the commands used to build an image. Docker images consist of read-only layers. Build images automatically by reading he instructions from the Dockerfile. Each represents a Dockerfile instruction. Layers are stacked. Each layer it a result of t changes from the previous layer In the past, docker used a technology called Linux Containers or LXC to manage containers on Linux. LXC used capabilities of the Linux Kernel such as Namespaces and CGroups, to create isolated environments on Linux known as containers. Problem: LXC is linux-specific, which was from linux open source project. Whenever there was a new release of linux kernel and changes in the lxc would impact docker containers. It was hard to work with LXC Directly for a normal user, that’s where docker provides a set of tools that made managing containers simpler. Docker Inc. developed their own tool called libcontainer as a replacement for LXC With the release of V0.9, docker introduced its own execution environment known as lib containers. With lib containers, docker could now directly interact with the linux kernel features such as namespaces and cgroups, and thus libcontainer replaced LXC as the default execution environments of Docker. Docker Daemon: THe docker eliend Interacts with the Docker deeme that perform running heavy lifting of building and distributing od focker containers Docker Client: It is the primary path of Docker users to interact with the Docker application. The client sends these commands to dockerd. From 1.11 they introduced 3 more components called ENTRYPOINT: Allows for configuring a container that will run as an executable ENTRYPOINT: Allows for configuring a container that will run as an executable VOLUME: Creates a mount point with the specified name and marks it as holding externally mounted volumes from native host or other containers USER: Sets the user name (or UID) and optionally the user group (or GID) to use when running the image and for any RUN, CMD, and ENTRYPOINT instructions that follow it in the Dockerfile WORKDIR: Sets the working directory for any RUN, CMD, ENTRYPOINT, COPY, and ADD instructions that follow it in the Dockerfile ARG: Defines a variable that users can pass at build-time to the builder with the docker build command, using the-build-arg <varname> <value> flag ONBUILD: Adds a trigger instruction to the image that will be executed at a later time, when the image is used as the base for another build March 4, 2023 0 comment 0 FacebookTwitterPinterestEmail
DevOPs Git Commands by Editorial Staff February 26, 2023 written by Editorial Staff How to create a new branch in Git: By default, you have one branch, the main branch. With this command, you can create a new branch. Git won’t switch to it automatically – you will need to do it manually with the next comman git branch branch_name root@vmlinux:/var/lib/git/.git/devops1# git branch br_work1 root@vmlinux:/var/lib/git/.git/devops1# git branch br_work2 root@vmlinux:/var/lib/git/.git/devops1# git branch br_work3 How to list branches in Git: You can view all created branches using the git branch command. It will show a list of all branches and mark the current branch with an asterisk and highlight it in green. git branch root@vmlinux:/var/lib/git/.git/devops1# git branch br_work1 br_work2 br_work3 * main How to switch to a newly created branch in Git: When you want to use a different or a newly created branch you can use this command git checkout branch_name root@vmlinux:/var/lib/git/.git/devops1# git checkout br_work1 Switched to branch 'br_work1' How to create a branch in Git and switch to it immediately: In a single command, you can create and switch to a new branch right away. git checkout -b branch_name root@vmlinux:/var/lib/git/.git/devops1# git checkout -b new_branch5 Switched to a new branch 'new_branch5' root@vmlinux:/var/lib/git/.git/devops1# git branch br_work1 br_work2 br_work3 main * new_branch5 How to delete a branch in Git: When you are done working with a branch and have merged it, you can delete it using the command below git branch -d branch_name root@vmlinux:/var/lib/git/.git/devops1# git branch -d br_work3 Deleted branch br_work3 (was 6f76b3f). How to merge two branches in Git: To merge the history of the branch you are currently in with the branch_name, you will need to use the command below git merge branch_name root@vmlinux:/var/lib/git/.git/devops1# git branch br_work1 br_work2 main * new_branch5 root@vmlinux:/var/lib/git/.git/devops1# git merge br_work1 Already up to date. root@vmlinux:/var/lib/git/.git/devops1# git merge br_work2 Already up to date. How to show the commit log as a graph in Git: We can use –graph to get the commit log to show as a graph. Also, –oneline will limit commit messages to a single line. git log –graph –oneline root@vmlinux:/var/lib/git/.git/devops1# git log --graph --oneline 6f76b3f (HEAD -> new_branch5, main, br_work2, br_work1) commiting some text files *f21baa3 (origin/main, origin/HEAD) added line in demo.txt *8c723ac info1 renamed to demo *98e404b created Python_1 folder and moved py files in that folder *5e55c5a Python Practise files uploaded *3e7a588 first 2 files created How to show the commit log as a graph of all branches in Git: Does the same as the command above, but for all branches git log –graph –oneline –all How to abort a conflicting merge in Git: If you want to throw a merge away and start over, you can run the following command: git merge –abort How to see remote URLs in Git: You can see all remote repositories for your local repository with this command git remote -v root@vmlinux:/var/lib/git/.git/devops1# git remote -v origin https://github.com/yvsc369/devops1 (fetch) origin https://github.com/yvsc369/devops1 (push) How to get more info about a remote repo in Git: Just replace origin with the name of the remote obtained by running the git remote -v command git remote show origin root@vmlinux:/var/lib/git/.git/devops1# git remote show origin remote origin Fetch URL: https://github.com/yvsc369/devops1 Push URL: https://github.com/yvsc369/devops1 HEAD branch: main Remote branch: main tracked Local branch configured for 'git pull': main merges with remote main Local ref configured for 'git push': main pushes to main (fast-forwardable) How to push changes to a remote repo in Git: When all your work is ready to be saved on a remote repository, you can push all changes using the command below git push while pushing you will get some type of errors like below root@vmlinux:/var/lib/git/.git/devops1# git push Username for 'https://github.com': yvsc369 Password for 'https://yvsc369@github.com': remote: Support for password authentication was removed on August 13, 2021. remote: Please see https://docs.github.com/en/get-started/getting-started- with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication. fatal: Authentication failed for 'https://github.com/yvsc369/devops1/ Method 1:- Create Public keys and Create SSH Keys in https://github.com/settings/ssh/new root@vmlinux:/var/lib/git/.git/devops1# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:0bo3C2Q4WlrLrFeQnsR3nRYwO3x2dZHBrwfi3/g2zcU root@vmlinux The key's randomart image is: +---[RSA 2048] + | o. .o*| | o o. +.| | . .. =.oo. .| | =..o.+=. | | o=+S. o . + | | Bo=. E| | o +.o o . =o| | .. o o o.=| | .. . oo| +----[SHA256] + root@vmlinux:/var/lib/git/.git/devops1# ls -l ~/.ssh/ total 8 -rw------- 1 root root 0 Sep 30 05:45 authorized_keys -rw------- 1 root root 1675 Sep 30 09:53 id_rsa -rw-r--r-- 1 root root 394 Sep 30 09:53 id_rsa.pub root@vmlinux:/var/lib/git/.git/devops1# cat ~/.ssh/id_rsa.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZaPMR0RofHfmlREo7ymKczXGZ7Hsw+wQ85XGUxd3FgZ4ZG 5ojXIHcfFk1Uk2eOSJ0W5tCtLLp/2xPCitr4Nx+Iex/zzeDdCZ2f68zfIkur0B0ZqgSs0Tn2JJTMv WmtS/HymOeJ5dPyOPiL/zEj8Nd28QQcD7c9BS410yCLV2pPdKGxo6VqQHPJn9OJvyvybLcm3wQoNP Ya1bEKNRURNYNv96cQDl+unOf1ymw+vHJvISMJhQx63gvV6Fr4j4sk3gP4y6sY62gHC1fUvgvgjF9 WfhlI1mmQCUHK19jHqvNv1Hd0KM8gPkN5GuP6ThO1deyorNbJQ76KCb45zCXqLF7 Copy the Key Code in Key Section box and add SSH Key mentioned below, and then try to push to github. Method 2:- Using Access Tokens Link to create token https://github.com/settings/tokens Kept for 90 days, you select all options as of now, because we are the beginners and we are at training stage root@vmlinux:/var/lib/git/.git/devops1# git push Username for 'https://github.com': yvsc369 Password for 'https://yvsc369@github.com': ghp_gco8ld6uLMx…DV (token) Counting objects: 3, done. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 5.00 MiB | 5.25 MiB/s, done. Total 3 (delta 0), reused 0 (delta 0) To https://github.com/yvsc369/devops1 f21baa3..6f76b3f main -> main How to add a remote repository in Git This command adds a remote repository to your local repository (just replace https://repo_here with your remote repo URL) git remote add origin git@github.com:yvsc369/devops1.git root@vmlinux:/var/lib/git/.git/devops1# git remote add origin git@github.com:yvsc369/devops1.git How to pull changes from a remote repo in Git: If other team members are working on your repository, you can retrieve the latest changes made to the remote repository with the command below git pull root@vmlinux:/var/lib/git/.git# git pull remote: Enumerating objects: 35, done. remote: Counting objects: 100% (35/35), done. remote: Compressing objects: 100% (31/31), done. remote: Total 35 (delta 9), reused 23 (delta 3), pack-reused 0 Unpacking objects: 100% (35/35), done. From github.com:yvsc369/devops1 * [new branch] main -> origin/main There is no tracking information for the current branch. Please specify which branch you want to merge with. See git-pull(1) for details. git pull <remote> <branch> If you wish to set tracking information for this branch you can do so with: git branch --set-upstream-to=origin/<branch> master How to check remote branches that Git is tracking: This command shows the name of all remote branches that Git is tracking for the current repository git branch -r root@vmlinux:/var/lib/git/.git# git branch -r origin/main How to fetch remote repo changes in Git: This command will download the changes from a remote repo but will not perform a merge on your local branch (as git pull does that instead). git fetch root@vmlinux:/var/lib/git/.git# git fetch root@vmlinux:/var/lib/git/.git# How to check the current commits log of a remote repo in Git Commit after commit, Git builds up a log. You can find out the remote repository log by using this command git log origin/main root@vmlinux:/var/lib/git/.git# git log origin/main commit 6f76b3faf315ba6b1cfa301fc4ff3f3549af6f93 (HEAD -> master, origin/main) Author: Venkat Chandra <yvsc369@gmail.com> Date: Fri Sep 30 07:51:02 2022 +0000 commiting some text files commit f21baa39ad597977928f8e678abd9b13b3e481f7 Author: Venkat Chandra <yvsc.369@hotmail.com> Date: Tue Sep 27 12:46:20 2022 +0530 added line in demo.txt commit 8c723ac023528cd3d91404d80a22275628d32433 Author: Venkat Chandra <yvsc.369@hotmail.com> Date: Tue Sep 27 12:30:21 2022 +0530 info1 renamed to demo How to merge a remote repo with your local repo in Git: If the remote repository has changes you want to merge with your local, then this command will do that for you git merge origin/main How to get the contents of remote branches in Git without automatically merging This lets you update the remote without merging any content into the local branches. You can call git merge or git checkout to do the merge. git remote update How to push a new branch to a remote repo in Git: If you want to push a branch to a remote repository you can use the command below. Just remember to add -u to create the branch upstream git push -u origin branch_name root@vmlinux:/var/lib/git/.git/devops1# git push -u origin new_branch5 Username for 'https://github.com': yvsc369 Password for 'https://yvsc369@github.com': Counting objects: 4, done. Compressing objects: 100% (3/3), done. Writing objects: 100% (4/4), 1002 bytes | 1002.00 KiB/s, done. Total 4 (delta 0), reused 0 (delta 0) remote: remote: Create a pull request for 'new_branch5' on GitHub by visiting: remote: https://github.com/yvsc369/devops1/pull/new/new_branch5 remote: To https://github.com/yvsc369/devops1 * [new branch] new_branch5 -> new_branch5 Branch 'new_branch5' set up to track remote branch 'new_branch5' from 'origin'. How to remove a remote branch in Git: If you no longer need a remote branch you can remove it using the command below git push –delete origin branch_name root@vmlinux:/var/lib/git/.git/devops1# git push --delete origin new_branch5 To https://github.com/yvsc369/devops1 - [deleted] new_branch5 How to use Git rebase: You can transfer completed work from one branch to another using git rebase. git rebase branch_name_here Git Rebase can get really messy if you don’t do it properly. Before using this command I suggest that you re-read the official documentation here How to run rebase interactively in Git: You can run git rebase interactively using the -i flag. It will open the editor and present a set of commands you can use. git rebase -i master # p, pick = use commit # r, reword = use commit, but edit the commit message # e, edit = use commit, but stop for amending # s, squash = use commit, but meld into previous commit # f, fixup = like “squash”, but discard this commit’s log message # x, exec = run command (the rest of the line) using shell # d, drop = remove commit How to force a push request in Git This command will force a push request. This is usually fine for pull request branches because nobody else should have cloned them. But this isn’t something that you want to do with public repos. git push -f root@vmlinux:/var/lib/git/.git/dev1/devops1# git checkout main Switched to branch 'main' Your branch is up to date with 'origin/main'. root@vmlinux:/var/lib/git/.git/dev1/devops1# git push -f Username for 'https://github.com': yvsc369 Password for 'https://yvsc369@github.com': TokenPassword Everything up-to-date February 26, 2023 0 comment 0 FacebookTwitterPinterestEmail
DevOPs Git Commands by Editorial Staff February 26, 2023 written by Editorial Staff How to add only certain files to the staging area in Git With the asterisk in the command below, you can add all files starting with ‘file’ in the staging area. git add file* root@vmlinux:/var/lib/git/.git/devops1# git add work* root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: work1 new file: work2 new file: work3 How to check a repository’s status in Git: This command will show the status of the current repository including staged, unstaged, and untracked files git status root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: 5MB.zip new file: filename1.txt new file: text1.txt new file: text2.txt new file: text3.txt How to commit changes in the editor in Git: This command will open a text editor in the terminal where you can write a full commit message. A commit message is made up of a short summary of changes, an empty line, and a full description of the changes after it. git commit How to commit changes with a message in Git: You can add a commit message without opening the editor. This command lets you only specify a short summary for your commit message. root@vmlinux:/var/lib/git/.git/devops1# git commit -m "committing some text files" [main 6f76b3f] commiting some text files 5 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 5MB.zip create mode 100644 filename1.txt create mode 100644 text1.txt create mode 100644 text2.txt create mode 100644 text3.txt How to commit changes (and skip the staging area) in Git: You can add and commit tracked files with a single command by using the -a and -m options. git commit -a -m “your commit message here” root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) modified: demo.txt modified: info.txt deleted: text1.txt renamed: filename1.txt -> updatefile1.txt Untracked files: (use "git add <file>..." to include in what will be committed) work1 work2 work3 root@vmlinux:/var/lib/git/.git/devops1# git commit -a -m "committing all workfiles" [main f336f12] commiting all workfiles files changed, 6 insertions(+), 3 deletions(-) delete mode 100644 text1.txt rename filename1.txt => updatefile1.txt (100%) How to see your commit history in Git: This command shows the commit history for the current repository: git log root@vmlinux:/var/lib/git/.git/devops1# git log commit 6f76b3faf315ba6b1cfa301fc4ff3f3549af6f93 (HEAD -> main) Author: Venkat Chandra <yvsc369@gmail.com> Date: Fri Sep 30 07:51:02 2022 +0000 commiting some text files commit f21baa39ad597977928f8e678abd9b13b3e481f7 (origin/main, origin/HEAD) Author: Venkat Chandra <yvsc.369@hotmail.com> Date: Tue Sep 27 12:46:20 2022 +0530 added line in demo.txt commit 8c723ac023528cd3d91404d80a22275628d32433 Author: Venkat Chandra <yvsc.369@hotmail.com> Date: Tue Sep 27 12:30:21 2022 +0530 info1 renamed to demo How to see your commit history including changes in Git: This command shows the commit’s history including all files and their changes git log -p root@vmlinux:/var/lib/git/.git/devops1# git log -p commit 6f76b3faf315ba6b1cfa301fc4ff3f3549af6f93 (HEAD -> main) Author: Venkat Chandra <yvsc369@gmail.com> Date: Fri Sep 30 07:51:02 2022 +0000 commiting some text files diff --git a/5MB.zip b/5MB.zip new file mode 100644 index 0000000..38da09a Binary files /dev/null and b/5MB.zip differ diff --git a/filename1.txt b/filename1.txt new file mode 100644 index 0000000..38da09a Binary files /dev/null and b/filename1.txt differ diff --git a/text1.txt b/text1.txt new file mode 100644 index 0000000..38da09a Binary files /dev/null and b/text1.txt differ diff --git a/text2.txt b/text2.txt new file mode 100644 index 0000000..38da09a Binary files /dev/null and b/text2.txt differ diff --git a/text3.txt b/text3.txt new file mode 100644 index 0000000..38da09a Binary files /dev/null and b/text3.txt differ How to see a specific commit in Git: This command shows a specific commit. Replace commit-id with the id of the commit that you find in the commit log after the word commit. git show commit-id root@vmlinux:/var/lib/git/.git/devops1# git show commit f336f12ccff6c48c2ecc11cf00eeafa53432d8ec (HEAD -> main, origin/main, origin/HEAD) Author: Venkat Chandra <yvsc369@gmail.com> Date: Fri Sep 30 13:23:18 2022 +0000 commiting all workfiles diff --git a/demo.txt b/demo.txt index 0757846..52228ef 100644 --- a/demo.txt +++ b/demo.txt How to see log stats in Git: This command will cause the Git log to show some statistics about the changes in each commit, including line(s) changed and file names. git log –stat root@vmlinux:/var/lib/git/.git/devops1# commit 6f76b3faf315ba6b1cfa301fc4ff3f3549af6f93 (HEAD -> main) Author: Venkat Chandra <yvsc369@gmail.com> Date: Fri Sep 30 07:51:02 2022 +0000 commiting some text files 5MB.zip | Bin 0 -> 5242880 bytes filename1.txt | Bin 0 -> 5242880 bytes text1.txt | Bin 0 -> 5242880 bytes text2.txt | Bin 0 -> 5242880 bytes text3.txt | Bin 0 -> 5242880 bytes files changed, 0 insertions(+), 0 deletions(-) How to see changes made before committing them using “diff” in Git: You can pass a file as a parameter to only see changes on a specific file. git diff shows only unstaged changes by default.We can call diff with the –staged flag to see any staged changes. git diff git diff all_checks.py git diff –staged How to see changes using “git add -p”: This command opens a prompt and asks if you want to stage changes or not, and includes other options. git add -p root@vmlinux:/var/lib/git/.git/devops1# git add -p diff --git a/demo.txt b/demo.txt index 0757846..52228ef 100644 --- a/demo.txt +++ b/demo.txt @@ -1,2 +1,3 @@ -i am vc from dba team -Point1 --Desc ---some dummy data1 \ No newline at end of file +i am vc from dba team +i am venkat from JavaHOME +Point1 --Desc ---some dummy data1 Stage this hunk [y,n,q,a,d,e,?]? y <stdin>:10: trailing whitespace. i am venkat from JavaHOME warning: 1 line adds whitespace errors. diff --git a/info.txt b/info.txt index f6bdfbf..a5a7307 100644 --- a/info.txt +++ b/info.txt @@ -1,3 +1,5 @@ hello devops1 training started -by venkat +info1 +info2 +info3 hello demo1 Stage this hunk [y,n,q,a,d,e,?]? y <stdin>:10: trailing whitespace. info3 warning: 1 line adds whitespace errors. How to remove tracked files from the current working tree: This command expects a commit message to explain why the file was deleted. git rm filename root@vmlinux:/var/lib/git/.git/devops1# git rm filename1.txt rm 'filename1.txt' Verify where the filename1.txt file is deleted or not ? root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) modified: demo.txt deleted: filename1.txt modified: info.txt Untracked files: (use "git add <file>..." to include in what will be committed) How to rename files in Git: This command stages the changes, then it expects a commit message git mv oldfile newfile root@vmlinux:/var/lib/git/.git/devops1# git mv textt1.txt updatefile1.txt root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) modified: demo.txt modified: info.txt deleted: text1.txt renamed: filename1.txt -> updatefile1.txt How to ignore files in Git: Create a gitignore file and commit it. How to revert unstaged changes in Git: git checkout filename How to revert staged changes in Git: You can use the -p option flag to specify the changes you want to reset. git reset HEAD filename git reset HEAD -p How to amend the most recent commit in Git: git commit –amend allows you to modify and add changes to the most recent commit. git commit –amend !!Note!!: fixing up a local commit with amend is great and you can push it to a shared repository after you’ve fixed it. But you should avoid amending commits that have already been made public. How to rollback the last commit in Git: git revert will create a new commit that is the opposite of everything in the given commit. We can revert the latest commit by using the head alias like this: git revert HEAD How to rollback an old commit in Git: You can revert an old commit using its commit id. This opens the editor so you can add a commit message. git revert comit_id_here How to create a new branch in Git: February 26, 2023 0 comment 0 FacebookTwitterPinterestEmail
DevOPs Git Commands by Editorial Staff February 26, 2023 written by Editorial Staff Installing on macOS There are several ways to install Git on a Mac. The easiest is probably to install the Xcode Command Line Tools. On Mavericks (10.9) or above you can do this simply by trying to run git from the Terminal the very first time $ git --version If you don’t have it installed already, it will prompt you to install it. If you want a more up to date version, you can also install it via a binary installer. A macOS Git installer is maintained and available for download at the Git website, at https://git- scm.com/download/mac. Installing on Windows There are also a few ways to install Git on Windows. The most official build is available for download on the Git website. Just go to https://git-scm.com/download/win and the download will start automatically. To verify whether its installed or not Note:- that this is a project called Git for Windows, which is separate from Git itself; for more information on it, go to https://gitforwindows.org. To get an automated installation you can use the Git Chocolatey package. Note that the Chocolatey package is community maintained. You can view all of your settings and where they are coming from using: $ git config –list –show-origin root@vmlinux:/var/lib/git# pwd /var/lib/git root@vmlinux:/var/lib/git# git config --list --show-origin file:.git/config core.repositoryformatversion=0 file:.git/config core.filemode=true file:.git/config core.bare=false file:.git/config core.logallrefupdates=true root@vmlinux:/var/lib/git# Your Identity root@vmlinux:/var/lib/git# git config --global user.name "Venkat Chandra" root@vmlinux:/var/lib/git# git config --global user.email yvsc369@gmail.com $ git config –list –show-origin root@vmlinux:/var/lib/git# git config --list --show-origin file:/root/.gitconfig user.name=Venkat Chandra file:/root/.gitconfig user.email=yvsc369@gmail.com file:.git/config core.repositoryformatversion=0 file:.git/config core.filemode=true file:.git/config core.bare=false file:.git/config core.logallrefupdates=true How to check your Git configuration: The command below returns a list of information about your git configuration including user name and email git config -l root@vmlinux:/var/lib/git# git config -l user.name=Venkat Chandra user.email=yvsc369@gmail.com core.repositoryformatversion=0 core.filemode=true core.bare=false core.logallrefupdates=true How to cache your login credentials in Git: You can store login credentials in the cache so you don’t have to type them in each time. Just use this command git config –global credential.helper cache How to initialize a Git repo: Everything starts from here. The first step is to initialize a new Git repo locally in your project root. You can do so with the command below git init root@vmlinux:/var/lib/git# git init Reinitialized existing Git repository in /var/lib/git/.git/ Cloning a sample project root@vmlinux:/var/lib/git/.git# git clone https://github.com/yvsc369/devops1 Cloning into 'devops1'... remote: Enumerating objects: 32, done. remote: Counting objects: 100% (32/32), done. remote: Compressing objects: 100% (28/28), done. remote: Total 32 (delta 8), reused 21 (delta 3), pack-reused 0 Unpacking objects: 100% (32/32), done. root@vmlinux:/var/lib/git/.git# ls -l total 40 -rw-r--r-- 1 root root 23 Sep 30 06:42 HEAD drwxr-xr-x 2 root root 4096 Sep 30 06:42 branches -rw-r--r-- 1 root root 92 Sep 30 06:56 config -rw-r--r-- 1 root root 73 Sep 30 06:42 description drwxr-xr-x 4 root root 4096 Sep 30 07:11 devops1 drwxr-xr-x 2 root root 4096 Sep 30 06:42 hooks drwxr-xr-x 2 root root 4096 Sep 30 06:42 info drwxr-xr-x 3 root root 4096 Sep 30 07:08 jhc01 drwxr-xr-x 4 root root 4096 Sep 30 06:42 objects drwxr-xr-x 4 root root 4096 Sep 30 06:42 refs Create some files like text1, text2 and text3 root@vmlinux:/var/lib/git/.git/devops1# ls -lrh total 26M -rw-r--r-- 1 root root 5.0M Sep 30 07:33 text3.txt -rw-r--r-- 1 root root 5.0M Sep 30 07:33 text2.txt -rw-r--r-- 1 root root 5.0M Sep 30 07:33 text1.txt -rw-r--r-- 1 root root 55 Sep 30 07:11 info.txt -rw-r--r-- 1 root root 5.0M Sep 30 07:34 filename1.txt -rw-r--r-- 1 root root 56 Sep 30 07:11 demo.txt drwxr-xr-x 2 root root 4.0K Sep 30 07:11 Python_1 -rw-r--r-- 1 root root 5.0M Jun 2 2008 5MB.zip root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Untracked files: (use "git add <file>..." to include in what will be committed) 5MB.zip filename1.txt text1.txt text2.txt text3.txt nothing added to commit but untracked files present (use "git add" to track) root@vmlinux:/var/lib/git/.git/devops1# How to add a file to the staging area in Git: The command below will add a file to the staging area. Just replace filename1.txt with the name of the file you want to add to the staging area. git add filename1.txt root@vmlinux:/var/lib/git/.git/devops1# git add filename1.txt root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: filename1.txt Untracked files: (use "git add <file>..." to include in what will be committed) 5MB.zip text1.txt text2.txt text3.txt How to add all files in the staging area in Git If you want to add all files in your project to the staging area, you can use a wildcard . and every file will be added for you. git add root@vmlinux:/var/lib/git/.git/devops1# git add . root@vmlinux:/var/lib/git/.git/devops1# git status On branch main Your branch is up to date with 'origin/main'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: 5MB.zip new file: filename1.txt new file: text1.txt new file: text2.txt new file: text3.txt February 26, 2023 0 comment 0 FacebookTwitterPinterestEmail
