Python

Static typing

https://docs.python.org/3/library/typing.html

Global Interpreter lock (GIL)

The GIL prevents true parallelism on multi-core processors, by allowing only one thread to run at a time.

https://wiki.python.org/moin/GlobalInterpreterLock

Synchronization

https://hackernoon.com/synchronization-primitives-in-python-564f89fee732

Web concepts

www sub domain

Ever wondered about the "www" in urls ? usually, we can give it a miss. so whats its use ? Lets say our site is milunsagle.in
1. if we have subdomains like ftp.milunsagle.in, music.milunsagle.in ,etc, and wish to keep cookies betwen the main domain and the subdomains separate, the www acts like a subdomain placeholder for the main domain. Else cookies for all will be mixed.
2. In DNS entries, which link the domain name to the server, if we have a www, we can point to a host-name rather than a static i.p, and the host name can then further point to multiple ips for redundancy.
thanks to https://www.sitepoint.com/domain-www-or-no-www/

Software concepts

Concurrency

Common concurrency terms explained :
http://doc.akka.io/docs/akka/current/java/general/terminology.html

 

 Normalization in short :

1NF : Ensure atomic(not a collection of) values in a column,
Ensure values of same data-type in a column

2NF : Remove partial dependencies : i.e. a column not depending on all the columns forming the primary key, but on some of them.

3NF: Remove transitive dependencies, i.e. a column not depending directly on the the primary key,, but on another column that depends on the primary key.

Spark

Spark basics : https://spark.apache.org/docs/1.4.0/programming-guide.html

 

 

Microservices

 

Introduction to microservices by Martin Fowler
https://martinfowler.com/microservices/

 

Image conversion tools

Imagemagick

 

Imagemagick is a great free tool to work with images. Below is a simple command to concatenate/combine images :

//+append for horz, - for vert
convert a.jpg b.jpg -append c.jpg

https://www.imagemagick.org/script/index.php

Raspberry Pi experiments

Connecting a TV

Wow, at last able to get the Rpi's video and audio working on my tv. It was not working even after the config.txt changes to turn off the HDMI. did the following do it ? Or was it just a pin connection issue ? dunno, but happy !

/opt/vc/bin/tvservice
o
/opt/vc/bin/tvservice -c "PAL 4:3"

UPDATE : 13-Nov-17
==================
I enabled the desktop mode using raspi-config and rebooted.
Did not need to execute above commands for tvservice.
Most importantly, i had to use the RED cable for video, even after using
the TRRS converter for the AV cable

------

To connect to pi over ssh using GUI, we do :
ssh
X pi@MyPiHostIP"
then
"pi@raspberrypi:~$ /etc/X11/Xsession"
This will start the X session on ur dektop

-------

When directly connected to the pi, it usually goes into command-prompt mode, and we can start a GUI session using "startx" command


Accessing the intranet 

Was unable to connect to my Pi from my Linux PC, both being on WLAN using tplink router. it was a router config issue. Under Wireless, Wireless Advanced, we have to turn off "Enable Client isolation" which is on by default.

Pi as a dumb(?) terminal

Using the raspberry pi to connect to VNC server on another machine. this way, one can share a single machine betw multiple people. of course, the pi can itself be used as a standalone computer, and the latest model 3, which is 64 bit, costs only around 3k. so if you are looking for a cheap computer, especially for kids, the pi + keyboard,mouse + monitor may be the way to go, and its GPIO pins can do many other interesting things.

https://www.raspberrypi.org/forums/viewtopic.php?f=48&t=41845

Raspberry Pi • View topic - VNC Client for RPI

raspberrypi.org

Keyboard

If keys are not working properly on the raspberry pi, esp the special keys, try changing the layout :

sudo vi /etc/default/keyboard
change the XKBLAYOUT propery to us:
XKBLAYOUT="us"

Linux often used commands

Editing

# diff
vimdiff gives a graphical feel

# Edit files with windows line endings
Pass the -b option to vi, or, once vi is loaded, type :e ++ff=unix.

# change the typing language, e.g. to hindi from English
alt + caps

Shell

# to make the statements/exports in a.sh available in current process
source a.sh

# search recursively for pattern
grep -r --include "*.jsp" pattern

# Sort files by reverse timestamp
find . -name "*.jsp" -printf '%T@ %t %p\n' | sort -k 1 -n -r

# line count
wc -l

FileSystem

# disk usage
df
du

Software management

# to invoke mint's package manager
sudo mintinstall

#to update flash plugin for FF
apt-get install flashplugin-installer

Services

# manage startup of services
initictl list/start/stop/restart <service>

#view/manage services
systemctl


# Remove a service from startup. Here apache
sudo update-rc.d apache2 remove

# Add a service apache2 back into the autostart

sudo update-rc.d apache2 defaults


# Enable autostart
sudo update-rc.d apache2 enable

# Disable autostart. Difference from remove is that the entry is kept.

update-rc.d apache2 disable 

#Can also be done with systemctl, depending on the version
#or the service command :
service apache2 start

Apache start/stop

sudo apachectl start/stop

# apache conf location
/etc/apache2/apache2.conf

Mail

postfix start/stop

d a-z or d * to delete mail messages

Processes

# view ports, in this case for amqp service
nmap -p 1-65535 localhost | grep mq

# view processes
ps -ef
netstat

Pdf

# pdftk remove password from pdf. may throw error, its outdated
pdftk input_pw <pass> in.pdf output out.pdf

# or use qpdf if pdftk gives error
qpdf --password=<your-password> --decrypt /path/to/secured.pdf out.pdf

Images

# image magic join images +for horz, - for vert
convert a.jpg b.jpg -append c.jpg

# image magic pdf to image
convert -density 100 -colorspace rgb test.pdf -scale 200x200 test.jpg

Md5

# check md5 sum of file. Tr to convert case
md5sum spark-2.2.1-bin-hadoop2.7.tgz | tr '[:lower:]' '[:upper:]' | grep C0081F6076070F0A6C6A607C71AC7E95

System Settings

Swap

There is a swappiness setting on ubuntu, which might make use of swap even when main memory is available. A lower value will prevent this
cat /proc/sys/vm/swappiness
sudo sysctl vm.swappiness=10

swapoff -av

swapon -av to reset any already used swap

Remote

# shutdown/restart remote PC over remmima
shutdown /s /t 0
shutdown /r /t 0

teamviewer --daemon stop

Tools utilities

ffmpeg

Cutting a section of an mp3 file on linux with ffmpeg. To copy a section of 30 sec from starting point of 15.5 seconds, use
ffmpeg -t 30 -ss 00:00:15.500 -i inputfile.mp3 -acodec copy outputfile.mp3

Free schema and ER tools :

https://www.leedsbeckett.ac.uk/qsee/

QSEE Technologies | CASE and Modeling tool specialists - Leeds Beckett University

leedsbeckett.ac.uk

Remote connection

Connected to my standby windows7 PC from my linux mint desktop, using Remmina.Yo ! Linux rocks !

http://www.digitalcitizen.life/connecting-windows-remote-desktop-ubuntu

Binary editors

xxd is a useful tool to convert to/from hex format. it can be used to feed binary data to text/newline based programs like sed/cut. e.g to remove first X bytes of X+Y byte records in binary data, do :

xxd
c X+Y -ps | cut -c 2X+1| xxd -r -p

Another out-of-the-box tool that can be used for binary data is : bbe - binary block editor

Also a java tool  : https://sourceforge.net/projects/bistreameditor/

REST tools

RESTED is a postman like extension for firefox, to test REST APIs : https://addons.mozilla.org/en-US/firefox/addon/rested/

Browser Styles

This global dark style from user-styles is great ! Easier on the eyes. Use with the stylish plugin for FireFox :
https://userstyles.org/styles/31267/global-dark-style-changes-everything-to-dark 
UPDATE : The stylish plugin is said to snoop on you, collecting data. There are alternative plugins like stylus.
Another option is to use a plugin that allows you to execute JS on load, and inject your style sheets via that. e.g. the onload scripts extension for chrome : https://chrome.google.com/webstore/detail/onload-scripts/gddicpebdonjdnkhonkkfkibnjpaclok

Java Streams

Streams in java are somewhat like iterators, in that they provide an-element-at-a-time interface.

However, they can also be processed in parallel.

And operations like filter, map, reduce, collect can be applied.

Below is a example of how the code can be really succinct.
We read a text file, and group by the first column, producing counts per group in a few lines:

Map mresult = Files.readAllLines( Paths.get("/home/test/testaccess.log")).stream()
            .map( line-> line.substring(0, line.indexOf(" ") ) )
            .collect( Collectors.groupingBy( line->line, Collectors.counting()) );
        System.out.println( mresult);