Scripting and Shell Scripts in Web Application Development

Scripting languages for web development

Scripting languages are interpreted languages used to write programs that automate tasks and add dynamic functionality to web applications. Common examples include Python, PHP, Node.js, and Bash.

A basic Python script that responds to a web request:

# Simple Python CGI script
import cgi

print('Content-type: text/html')
print()
print('<h1>Hello from Python!</h1>')

A basic Node.js HTTP server:

const http = require('http');

http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/html' });
  res.end('<h1>Hello from Node.js!</h1>');
}).listen(3000);

Creating files and directories with shell scripts

Shell scripts automate file and directory management using commands like mkdir (make directory), touch (create file), and rm (remove).

Creating a directory

#!/bin/bash
# Create a new directory for a web project
mkdir my_web_project
echo 'Directory created successfully'

Creating files

#!/bin/bash
# Create project files
touch index.html style.css script.js
echo 'Project files created'

Setting up a full project structure

#!/bin/bash
# Automate web project setup
mkdir -p my_site/css my_site/js my_site/images
touch my_site/index.html
touch my_site/css/style.css
touch my_site/js/app.js
echo 'Project structure ready!'

The -p flag in mkdir creates parent directories as needed and avoids errors if the directory already exists.

Searching for text in a file

The grep command searches for patterns in text files and is one of the most commonly used tools in shell scripting for web applications.

Basic grep search

# Search for the word 'error' in a log file
grep 'error' server.log

Case-insensitive search

# -i flag makes the search case-insensitive
grep -i 'error' server.log

Show line numbers

# -n flag shows the line number of each match
grep -n 'error' server.log

Search recursively through a directory

# -r searches through all files in a folder
grep -r 'TODO' ./my_site/

Count matches

# -c counts the number of matching lines
grep -c 'error' server.log

Using awk and sed for text processing

awk and sed are powerful tools for extracting and transforming text data from files.

awk — extract a specific column

# Print the first column (username) from a CSV file
awk -F',' '{ print $1 }' users.csv

sed — find and replace text

# Replace 'http' with 'https' in a config file
sed 's/http/https/g' config.txt

The s/old/new/g syntax means: substitute 'old' with 'new', globally (on every occurrence in the line).

Integrating shell scripts with a web application

Shell scripts can be triggered by backend code (e.g. Python or Node.js) to run file operations as part of a web request.

Running a shell script from Python

import subprocess

# Run a shell script when a user submits a form
result = subprocess.run(['bash', 'setup.sh'], capture_output=True, text=True)
print(result.stdout)

Running a shell command from Node.js

const { exec } = require('child_process');

// Search a log file when a route is requested
exec('grep -i error server.log', (err, stdout) => {
  if (err) return console.error(err);
  console.log(stdout);
});