Self signed wildcard security certificate for apache or nginx

This tutorial is done on a debian 11 system… it should work for wildcard (For all subdomains under a domain), but also for subdomains or the primary domain, obviously, all you need to do is replace the * which denotes wildcard with the subdomain of your choice, so *.qworqs.com is wildcard, yazeed.qworqs.com is a subdomain 😉 so let us get started

Let’s encrypt has certainly revolutionized the world of SSL certificates (By making them free), but when it comes to wildcard certificates, let’s encrypt will require more than just generating the certificate, it will require a system that automatically alters DNS at your registrar, and differs from registrar to registrar.

So while I am developing, and need a wildcard SSL, I can simply generate a self signed wildcard security certificate, and teach my browser to accept it, and that is that, so here is how to generate that certificate !

So let us get started, first let us create a public and private key in one go, and a folder to store them !

cd /etc/ssl
sudo mkdir qworqs.com
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/qworqs.com/wildcard-ss.key -out /etc/ssl/qworqs.com/wildcard-ss.crt

Now that we have the public and private key, we also need a strong Diffie-Hellman group… this file though goes somewhere else in the nginx directory

sudo openssl dhparam -out /etc/nginx/dhparam.pem 4096

Now you are done with creating everything you need, the next step is to install them into your nginx configuration

this can be done directly in the website’s configuration files, since this is a wildcard certificate that is expected to be installed in multiple nginx files, it is a good idea to group then so that you can add one line to all the config files, this will serve you when the certificates expire and you need to renew them !

Types of documents in software development and who writes them and for who

Every company has it’s own procedures, and sometimes it’s own standard for the following documents, but this is the most common, the order is loosely relevant to prerequisites and chronological order

I have put them down in a table to simplify

In this document, a client refers to a party that receives the code, (any of the steakholders), Implementation lead and developer refer to the programmers, system analyst refers to a system analyst.

Doc
Abbrv.
Document descriptionWho writes itWho is it written for
SOW
Statement of workProject management, Chief Information Officer, third-party contractor, So from the developers perspective, any client such as the aboveImplementation Lead
MRDMarketing requirements documentMarketing departmentAll steakholders including Implementation lead
URD
URS
The user requirements document
User requirements specification

This document is basically the client outlining the features the developers are to implement
Project management with help from system analysts (Clients)Implementation lead and any relevant stakeholders
SRSA software requirements specification sheet
A description of a software system to be developed, laying out functional and non-functional requirements- or features
This document bridges the gap between user/client and developer
Also serves as an agreement
Business Analyst, System Analyst, and developers
TRDTechnical requirements document

Written by the developers based on the requirement documents submitted by the client, this is an extensive document that connects functionality, features, and purpose together, creating this document is a very lengthy process and it requires “Technical writing skills” as it is meant to convey the whole system to non technical steakholders.
FSDFunctional specification document
FRDFunctional Requirements Document
PRDProduct requirements document.

This document communicates the capabilities the product will need.
SRDSoftware Requirements Document
written statement of what the software will do or should do.
FRSFar more detailed than an SRSImplementation lead or System Analyst.
Product RoadmapTimetable
Product backlogIt is the prioritized list of task-level details needed to execute the strategic plan outlined in the product roadmap.
Sprint BacklogDrawn from the product backlog, this is the list of cross-functional team plans to work on in the next sprint.
SDSoftware documentation
A user’s manual (Not for the developers)
Documents relevant to software development

Laravel development under Linux (Dev Tools)

At this stage, this posst is a stub, I am simply compiling information here then I will come back to edit this

Under linux, the Laravel extensions that i like up to now are

  • Laravel Blade Snippets (For blade templates)
  • PHP namespace resolver (So that we don’t need to look for what namespaces provide for the classes we are using)
  • PHP Intelephense (PHP Autocomplete/intellisense ref finder, etc…)

Laravel 9 on NGINX under Debian 11 (Bullseye) – Step by step

This part of the tutorial is a hands on setup to have Laravel working on nginx under Linux. this is not the only way to run Laravel, for all the options, see here

NOTE: Most if not all of the popular development tools are available for Linux and Windows, so If this setup is for development, you might want to install Debian Linux with the Gnome GUI so that you can install the development tools used in the next part of this tutorial, if you are going to be developing on a different machine or this is for production, then you shouldn’t.

STEP 1: Install Debian 11 (Bullseye) on a computer.

STEP 2: Install the basic Laravel environment with the following commands, Laravel 9 requires PHP 8, which at the time of writing is not available in Debian 11 repositories, so we will need to add the repositories from the guy who manages PHP for Debian anyway

sudo apt-get install ca-certificates apt-transport-https software-properties-common gnupg unzip curl
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | tee /etc/apt/sources.list.d/sury-php.list
wget -qO - https://packages.sury.org/php/apt.gpg | apt-key add -
apt-get update

Now, to the actual installation of the environment

apt install php8.1-fpm nginx
apt install php8.1-{dev,common,xml,tokenizer,zip,mysql,curl,mbstring,mysql,opcache,gd,intl,xsl,bcmath,imap,soap,readline,sqlite3,gmp}
apt install redis-server
apt-get install php8.1-redis
apt install mariadb-server mariadb-client

Now you need to secure redis !

Now remember to secure your mariaDB (MySQL) installation with the following command

mysql_secure_installation

Next, we need composer, the PHP dependency manager, to get it execute the following

curl -sS https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer
chmod +x /usr/local/bin/composer
composer --version

The last line above should show you what version of composer you have just installed

Now, remember to never run composer as root, but rather as a regular user, from this point on I am assuming you are running the terminal as a regular user.

I understand that Debian puts the web root in /var/www/html, but i usually like to create a separate directory called /var/vhosts and put all my web projects in it

mkdir /var/vhosts

Now, we need to create a sample project for our learning and training experience

cd /var/vhosts
composer create-project laravel/laravel laraveltestapp
chmod -R 0777 /var/vhosts/laravelapp/storage

Next, We would need to setup nginx to serve this website (“Would like to” actually, because there are alternatives, but I’m keeping it simple), here is an almost standard template for nginx, modify the host name and project name to match your project and preferences.

NGINX vhost config file (In my case /etc/nginx/sites-available/laraveltestapp)

server {
    listen 80;
    listen [::]:80;
    server_name ltest.net www.ltest.net;
    root /var/vhosts/laraveltestapp/public;

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options "nosniff";

    index index.html index.htm index.php;

    charset utf-8;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    error_page 404 /index.php;

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php8.1-fpm.sock;
#	fastcgi_pass unix:/var/run/php-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location ~ /\.(?!well-known).* {
        deny all;
    }
}

Surely for this to work, you will need to add the following two lines to the end of the /etc/hosts file

127.0.0.1 ltest.net
127.0.0.1 www.ltest.net

As soon as we have the file above, we need to create a symbolic link for it in the sites enabled directory

cd /etc/nginx/sites-enabled
ln -s /etc/nginx/sites-available/laraveltestapp

Suddenly, it works through nginx, you should see the welcome page (View) here

Now, to the development environment, how to setup your development environment can be found here

Video Editing for Debian 11

On linux, when i needed to edit a video I usually go to Kdenlive, I used to think it is THE linux alternative to Adobe Premier Pro, until yesterday, when a friend recommended I try out OpenShot

I am a Gnome user, Kdenlive was designed for KDE, I have always run it in gnome (Unity) and it worked fine, but today, I am feeling adventurous and up for trying OpenShot, openshot too is pyQt which is a bummer, but hey, this is not why we doing this

Reminder
KDE is based on Qt and Gnome is based on GTK. both applications here are meant for KDE, and I don’t really want to install KDE on my machine

This post here is where I will leave my impressions so hang tight.

Debian 11 on laptop with nVidia optimus

This is a short one, just a quick referance to have your system optimized for both efficiency and performance

This laptop which has had a fresh Debian Bullseye installed has both an nVidia card alongside the intel card, to find out what cards your system is running, you could start with the command

sudo lspci

00:02.0 VGA compatible controller: Intel Corporation HD Graphics 620 (rev 02)
01:00.0 3D controller: NVIDIA Corporation GM108M [GeForce 940MX] (rev a2)

Or you could simply get the relevant data with

lshw -c video

This is a 7th generation intel CPU, namely the “Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz”

Let us start by installing the nVidia drivers from nvidia, in my case, i installed the detection script followed by nvidia-drivers package from the non-free repositories

If your CPU is post 2007, Make sure you do not install “xserver-xorg-video-intel”, if it is already installed remove it ! we want xserver-xorg-core to manage the intel graphics

Install nvidia primus

apt-get install primus

Once that is setup, to figure out which card is being used as the main one, you can run

glxinfo|egrep "OpenGL vendor|OpenGL renderer"

To execute the command with the nvidia card and check if it is being used, execute it in the following way

primusrun glxinfo|egrep "OpenGL vendor|OpenGL renderer"

Now, without the primusrun you should get intel, with the primusrun, you should see nvidia

Now, you know the system is taking care of what is running which application correctly, I will follow with more information once i have the time.

To instruct the system to use your nvidia card to play a video, you can execute something like

primusrun totem

(this will force it to use your nvidia card)

Another command that will show you the utilization of your nvidia GPU (Mine is mostly 0 percent as I am not running anything with a primusrun prefix)

nvidia-smi

Laravel – Tools and environments

The following are things that are commonly used with Laravel. broken down by category

Environment tools and solutions

  • PHP composer (Mandatory)
  • Docker – Sail (Optional)

Relevant / Compatible server side software

  • Apache2 (2.4) (Web Server)
  • Nginx (Web Server)
  • PHP (PHP8.x for Laravel 9)
  • redis (Data structure store)
  • memcache (in-memory, distributed cache)
  • MeiliSearch (Search application)
  • MailHog (email-testing tool with a fake SMTP server)
  • Selinium (browser automation)
  • MySQL/Mariadb (Database engine, most common)
  • PostgreSQL (Database Engine, very powerful)

Development environments

  • Microsoft Visual Studio Code (Free): built-in support for JavaScript, TypeScript and Node.js, extensions include PHP
  • Sublime text (Not free, works but nags)
  • Apache beans (Free)
  • IntellijIDEA (Not free, one month trial)
  • PHPStorm (Same as IntellijIDEA without the other languages support, Not Free, one month trial)

Database Management

  • MAC: Sequel Pro and sequel ace
  • ALL: PHPMyAdmin, mysql workbench (Oracle),
  • sqlyog (Windows and Linux)


Other

  • Node.js
  • NPM (Installed with node.js): npm is a package manager for the JavaScript to work with front end stuff like node.js, react.js, bootstrap, tailwind CSS,
  • Yarn (NPM alternative from Facebook)
  • GIT: track changes in source code

Downloading video from youtube

This might sound complicated, but youtube videos come in many shapes, many of them have either video only, or audio only !

the best way to get a full resolution video is to download then combine video and audio !

But before we go there, youtube throttles the speed you download videos at ! so youtube-dl needs patching, an alternative would be yt-dlp (See here)

Making sure things will install with pip correctly

apt install python3-pip

python3 -m pip install --force-reinstall https://github.com/yt-dlp/yt-dlp/archive/master.tar.gz

Now, let us try downloading a video/audio

youtube-dl -F https://www.youtube.com/watch?v=mUvrLxaSolc

Now, to combine them without re-encoding

ffmpeg -i ao.webm -i vo.webm -c:v copy -c:a copy output.webm

But remember when you download,

Only VP8 or VP9 or AV1 video and Vorbis or Opus audio and WebVTT subtitles are supported for WebM.

Moving away from Laravel smoothly (Pros and cons)

A short while back, I was handed a repository with code written in Laravel, incomplete, and somewhat sketchy. with the purpose of taking a look at the code and deciding whether i would take it or not.

To give you the lowdown FIRST, while first researching Laravel, I started by investigating the limitations, my first google search sent me in the direction of a blog post by Beau Beauchamp, a developer who seems familiar with the framework.

I obviously didn’t take his word for it, I don’t really know who he is, so i gave myself a 2 day intensive Laravel course, unfortunately he was somewhat right.

The following two paragraphs are from his blog post, they don’t tell you much, but are better explained as I go

Laravel prides itself as the framework for “artisans”. The impression is that Laravel is the framework for people who don’t really know how to code and don’t want to learn. I get it.

Laravel is not PHP, per se, it uses an “expressive” syntax or what has been coined as “syntactic sugar” to hide things from you that it thinks “artisans” don’t need to worry about.

Having no experience with Laravel, and plenty of experience in PHP, the 2 day course i mentioned earlier left me with the following impressions, I was Impressed by how massive it is (Implementing plenty of features with very few lines), impressed by how simple and easy it is (Truly made for people who don’t want to learn programming). and thinking that this is basically a great framework for a simple straight forward website, but once you are looking to give the website more edge, a competitive advantage, or creating complex functionality, the framework is pretty restrictive and not so scalable.

yes, caching can help with the scalability part, but the degree caching helps with depends on the nature of the website, and for this particular purpose, it is not a perfect solution.

So should we throw the existing code away ?

My answer is NO, if i do end up taking this job, I plan to launch with the Laravel code, then extend the software with good old plain old PHP, with the database acting as the API between the new system and the old system, this way, the website owner can have a functional website where he can promote and advertise, and dip his toes in the water while a different system slowly takes this system’s place as it gets developed.

After updating the existing code from Laravel 7 to Laravel 9 (Overhead), running a security audit for the code, A varnish or nginx proxy should sit in the middle, and new code should run transparently through the proxy, when that happens, I am not even restricted to the same virtual machine running Laravel, I can have 2 virtual machines running different tools acting as one website, transparently, without the user ever knowing.

The front end with react

The other issue I have with this project is with react and react-native, which are the front ends of both web and mobile applications.

React is a very cool framework, but there is quite a bit of controversy around it and around Ajax in general when it comes to Search Engine optimization (SEO), in a statement by google ten years ago, googlebot is now able to read a website the same way a web browser does, and I have seen that they do see it that way from ten years ago when they were providing tools telling people what pages on their websites were having horizontal scroll bars, but regardless of that statement, the fact that most websites that appear in my search results are not Ajax, and that HTML and CSS still run most of the popular websites does raise some concerns, entering into a very competitive market dictates that every inch of a competitive advantage is vital to our success.

So, first let me get the advantages and disadvantages of Laravel out of the way, then get into the technicalities and how the new system should co-exist with Laravel and react.

Pros

  • Laravel is a very mature framework, but very opinionated, opinionated means the designers of the framework expect you to create your website in one specific way, and as long as you are within those lines, you can make things work, what mature and popular means is that when you don’t see those lines, someone online has probably mentioned how to do it with Laravel.
  • Laravel is not the greatest in backwards compatibility, so when a new release comes out, it is not just PHP that you need to worry about, it is also Laravel, and from people’s experiences online, things tend to either break or become buggy when a major release of Laravel is out
  • Laravel is heavy, very heavy, and to deal with that, the developers have come up with workarounds, mainly caching, which lends itself to certain websites more than others, sometimes caching can have so little benefit, and sometimes it is a magic recipe for super snappy
  • Laravel is based on symphony, and works great with react, but even though google has claimed that their spider treats

WordPress does not load correctly (SOLVED) behind nginx/varnish reverse proxies !

Here is my problem, I have a website, and in a directory in that website, I have a wordpress installation, and that installation opens correctly, loads all the images, css, js and any other files for a proper experience, only problem is, when you put this behind a varnish reverse proxy, and an nginx reverse proxy for SSL (https), the website design (theme) does not load, you only see the actual html page that was loaded, but all other elements are never fetched from the server, i actually sniffed the data and found that css, javascript, and images are never even requested !

So the short of this story, if you are having problems with page loading without the theme or design, and you have a similar setup, odds are the problem is with wordpress settings not with nginx or with varnish !

So a closer look at the page source reveals that the page was loaded with https, but the links to all the page resources are in HTTP ! why is that ? simple

when you open the website in SSL, your browser creates a secure connection with nginx (termination), nginx requests the page from varnish, which relays the page again to the server.

As far as the web server serving wordpress is concerned, this request came in http, not https ! so all the page resources should be in http right ? yes, this is what is happening, but what is the solution

I tried a few solutions, for example, i changed the wordpress address and site address to httpS, but wordpress is smart enough to use whatever protocol the user accessed and use it for all resources !

there are many solutions programmatically, which is something i avoid because i update wordpress, and don’t want to fix it every time i upgrade, so whatever solution i need has got to be in the only file that is never modified when upgrading wordpress, the config file

wordpress knows it is on SSL from the following two entries in the environment, the entries $_SERVER[‘HTTPS’] and $_SERVER[‘SERVER_PORT’], the proxy sends a hint that the user used https with the variable $_SERVER[‘HTTP_X_FORWARDED_PROTO’] in the request header, hence, adding the following code snippet somewhere in the beginning of the config file should deceive wordpress into thinking it has been accessed over https !

if($_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https'){
   $_SERVER['HTTPS'] = 'on';
   $_SERVER['SERVER_PORT'] = 443;
 }

Hope this all works okay for you, if not please let me know in the comments and i would be more than glad to help