[{"content":"Recently, I have been building software without libc to better understand Linux syscalls and internals better. So far, I have built a minimal shell, terminal Snake game, pure ARM64 assembly HTTP server and threads implementation. I have been using strace extensively while debugging.\nUseful options and flags I use a version of the following command:\nstrace -fintrCDTYyy -o strace.log -v -s128 ./binary This looks like an alphabet soup of options! Here\u0026rsquo;s what they do and how they are useful:\n-f: Follow child processes/threads. This is especially useful when dealing with spawning processes or threads as otherwise, strace will only trace the parent process. -v: Print unabbreviated versions of environment, stat, termios and other structs in syscalls. I found this invaluable in conjunction with -s when doing assembly programming to check if the structs were being initialized correctly and if certain arguments were being sent in little/big endian format -s NUM: Specify the maximum string size to print. Useful for large structs -o: Save strace output to a log file. It is always better to do this to investigate the output of the original process and strace separately without each cluttering the other -yy: Print all available information associated with file descriptors. This is great for expanding the file descriptor to either its full path in case of a file or TCP address in case of sockets -Y: Print command names for PIDs. I found this useful when building the shell to check if the correct program is being executed -t: Print current timestamp in log -T: Show time spent in syscalls. Useful for some basic profiling although strace heavily slows down the process. -r: Print a relative timestamp upon entry to each system call -n: Print syscall number. Great to quickly find out syscall numbers on new architectures. -i: Print instruction pointer at the time of syscall. Found this useful when debugging assembly code to check rough location of errors. -C: Print summary of syscall count, time, errors at the end of regular output Print stack traces The -k or --stack-trace prints the stacktrace along with the syscall. This is useful if your program is compiled with with -g. This post is a good read on using strace to show backtraces for a Golang program compiled with GODEBUG.\nSelectively tracing syscalls By default, strace traces all syscalls. We can be very selective in tracing calls by using the -e option. It allows us to trace families of syscalls e.g -e t=%net will trace all network related syscalls while -e t=%mem will trace memory related syscalls.\nWe can also trace only syscalls which succeed using -z option or fail using -Z option. Another useful option is -P which can be used to only trace syscalls which access a particular path e.g strace -f -P /usr/bin/ls sh -c ls.\nTampering with syscalls! strace is very powerful and can do stuff like inject faults for particular syscalls e.g strace -e inject=%file:error=ENOENT:when=3+ ls which fails the all file related syscalls after 2 successful invocations. We can also make syscalls return a particular value using retval or send signal using signal or delay at beginning or end of syscall using delay_enter and delay_exit with -e inject.\nI found this useful when debugging failure cases in code for the syscalls. The strace output will have lines like openat(AT_FDCWD, \u0026quot;/lib/libc.so.6\u0026quot;, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) (INJECTED) which have INJECTED marked in them. This is useful to differentiate genuine syscall errors from the injected ones.\n","permalink":"https://rrampage.github.io/2025/06/13/strace-tips-for-better-debugging/","summary":"\u003cp\u003eRecently, I have been building software without libc to better understand Linux syscalls and internals better. So far, I have built a \u003ca href=\"https://gist.github.com/rrampage/5046b60ca2d040bcffb49ee38e86041f\"\u003eminimal shell\u003c/a\u003e, \u003ca href=\"https://gist.github.com/rrampage/2a781662645dc2fcba45784eb584cbdc\"\u003eterminal Snake game\u003c/a\u003e, pure ARM64 assembly \u003ca href=\"https://gist.github.com/rrampage/d31e75647a77badb3586ebae1e414cb6\"\u003eHTTP server\u003c/a\u003e and \u003ca href=\"https://gist.github.com/rrampage/43c56d4a56f8f73320d17ff7b3a49be6\"\u003ethreads implementation\u003c/a\u003e. I have been using \u003ca href=\"https://man7.org/linux/man-pages/man1/strace.1.html\"\u003e\u003ccode\u003estrace\u003c/code\u003e\u003c/a\u003e extensively while debugging.\u003c/p\u003e\n\u003ch2 id=\"useful-options-and-flags\"\u003eUseful options and flags\u003c/h2\u003e\n\u003cp\u003eI use a version of the following command:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003estrace -fintrCDTYyy -o strace.log -v -s128 ./binary\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eThis looks like an alphabet soup of options! Here\u0026rsquo;s what they do and how they are useful:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003e-f\u003c/code\u003e: Follow child processes/threads. This is especially useful when dealing with spawning processes or threads as otherwise, strace will only trace the parent process.\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-v\u003c/code\u003e: Print unabbreviated versions of environment, stat, termios and other structs in  syscalls. I found this invaluable in conjunction with \u003ccode\u003e-s\u003c/code\u003e when doing assembly programming to check if the structs were being initialized correctly and if certain arguments were being sent in little/big endian format\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-s NUM\u003c/code\u003e: Specify the maximum string size to print. Useful for large structs\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-o\u003c/code\u003e: Save strace output to a log file. It is always better to do this to investigate the output of the original process and strace separately without each cluttering the other\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-yy\u003c/code\u003e: Print all available information associated with file descriptors. This is great for expanding the file descriptor to either its full path in case of a file or TCP address in case of sockets\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-Y\u003c/code\u003e: Print command names for PIDs. I found this useful when building the shell to check if the correct program is being executed\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-t\u003c/code\u003e: Print current timestamp in log\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-T\u003c/code\u003e: Show time spent in syscalls. Useful for some basic profiling although strace heavily slows down the process.\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-r\u003c/code\u003e: Print a relative timestamp upon entry to each system call\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-n\u003c/code\u003e: Print syscall number. Great to quickly find out syscall numbers on new architectures.\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-i\u003c/code\u003e: Print instruction pointer at the time of syscall. Found this useful when debugging assembly code to check rough location of errors.\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e-C\u003c/code\u003e: Print summary of syscall count, time, errors at the end of regular output\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"print-stack-traces\"\u003ePrint stack traces\u003c/h2\u003e\n\u003cp\u003eThe \u003ccode\u003e-k\u003c/code\u003e or \u003ccode\u003e--stack-trace\u003c/code\u003e prints the stacktrace along with the syscall. This is useful if your program is compiled with with \u003ccode\u003e-g\u003c/code\u003e. \u003ca href=\"https://shane.ai/posts/backtraces-with-strace/\"\u003eThis post\u003c/a\u003e is a good read on using strace to show backtraces for a Golang program compiled with \u003ccode\u003eGODEBUG\u003c/code\u003e.\u003c/p\u003e","title":"strace tips for better debugging"},{"content":"Oops I deleted my bin dir is a fun set of challenges to investigate a \u0026ldquo;rogue\u0026rdquo; process which has been deleting files on the system. It has deleted all programs from the bin directories like ps, htop, lsof. All we have left is a shell prompt\u0026hellip;\nBash built-ins In the challenge, the shell we are using is bash. Bash has many \u0026ldquo;built-ins\u0026rdquo; which are commands we can run without having to spawn a process and execute a binary i.e they are part of bash itself.\nThe most useful one right now is help which lists all the bash built-ins and their arguments. Typing help BUILTIN_NAME will give more details about that built-in command.\nList of built-ins job_spec [\u0026amp;] history [-c] [-d offset] [n] or history -anrw [filename] or history -ps[1/218] (( expression )) if COMMANDS; then COMMANDS; [ elif COMMANDS; then COMMANDS; ]... [ else COM\u0026gt; . filename [arguments] jobs [-lnprs] [jobspec ...] or jobs -x command [args] : kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigs\u0026gt; [ arg... ] let arg [arg ...] [[ expression ]] local [option] name[=value] ... alias [-p] [name[=value] ... ] logout [n] bg [job_spec ...] mapfile [-d delim] [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callb\u0026gt; bind [-lpsvPSVX] [-m keymap] [-f filename] [-q name] [-u name] [-r keyseq] [\u0026gt; popd [-n] [+N | -N] break [n] printf [-v var] format [arguments] builtin [shell-builtin [arg ...]] pushd [-n] [+N | -N | dir] caller [expr] pwd [-LP] case WORD in [PATTERN [| PATTERN]...) COMMANDS ;;]... esac read [-ers] [-a array] [-d delim] [-i text] [-n nchars] [-N nchars] [-p pro\u0026gt; cd [-L|[-P [-e]] [-@]] [dir] readarray [-d delim] [-n count] [-O origin] [-s count] [-t] [-u fd] [-C cal\u0026gt; command [-pVv] command [arg ...] readonly [-aAf] [name[=value] ...] or readonly -p compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlist] [\u0026gt; return [n] complete [-abcdefgjksuv] [-pr] [-DEI] [-o option] [-A action] [-G globpat] [\u0026gt; select NAME [in WORDS ... ;] do COMMANDS; done compopt [-o|+o option] [-DEI] [name ...] set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...] continue [n] shift [n] coproc [NAME] command [redirections] shopt [-pqsu] [-o] [optname ...] declare [-aAfFgiIlnrtux] [-p] [name[=value] ...] source filename [arguments] dirs [-clpv] [+N] [-N] suspend [-f] disown [-h] [-ar] [jobspec ... | pid ...] test [expr] echo [-neE] [arg ...] time [-p] pipeline enable [-a] [-dnps] [-f filename] [name ...] times eval [arg ...] trap [-lp] [[arg] signal_spec ...] exec [-cl] [-a name] [command [argument ...]] [redirection ...] true exit [n] type [-afptP] name [name ...] export [-fn] [name[=value] ...] or export -p typeset [-aAfFgiIlnrtux] [-p] name[=value] ... false ulimit [-SHabcdefiklmnpqrstuvxPT] [limit] fc [-e ename] [-lnr] [first] [last] or fc -s [pat=rep] [command] umask [-p] [-S] [mode] fg [job_spec] unalias [-a] name [name ...] for NAME [in WORDS ... ] ; do COMMANDS; done unset [-f] [-v] [-n] [name ...] for (( exp1; exp2; exp3 )); do COMMANDS; done until COMMANDS; do COMMANDS; done function name { COMMANDS ; } or name () { COMMANDS ; } variables - Names and meanings of some shell variables getopts optstring name [arg ...] wait [-fn] [-p var] [id ...] hash [-lr] [-p pathname] [-dt] [name ...] while COMMANDS; do COMMANDS; done help [-dms] [pattern ...] { COMMANDS ; } Challenge 1 - print the current working directory Solution The `pwd` built-in shows the *p*resent *w*orking *d*irectory Challenge 2 - List all files in the current working directory We can not use ls as it is gone\u0026hellip; Which built-in can help us?\nSolution echo * This works because echo prints all its arguments to stdout. The * character is a wildcard character which makes bash perform glob expansion. This will expand the * to match all files in the directory.\nChallenge 3 - Display contents of a file No cat! We can still use echo to print stuff to stdout. How do we pass the file contents to echo?\nSolution echo \u0026#34;$(\u0026lt;file_name)\u0026#34; \u0026lt;file_name redirects the file contents to stdin. $(\u0026lt;file_name) is command substitution which means that bash executes the command in a sub-shell and substitutes the command with stdout of the command \u0026quot;$(\u0026lt;file_name)\u0026quot; - we wrap this in double quotes to prevent bash from expanding newlines and spaces Finally, we pass the whole string to echo Challenge 4 - Print the name of the running processes We can not use ps. /proc can be used as it is just a file system which can be read similar to how we displayed file contents earlier\nSolution for f in /proc/*/cmdline; do echo $(\u0026lt;$f); done /proc has directories for each pid. In the pid directory for each process, there will be files for: cmdline: this shows the command line arguments passed by Linux to the process comm: name of the process cwd: current working directory of the process i.e where it was started exe: symlink to the actual location of the binary being executed by the process status: More details about the process like memory usage, signal handlers, etc syscall: system call number and argument registers for the system call currently being executed by the process, followed by the values of the stack pointer and program counter registers /proc/*/cmdline will get cmdline file for all running processes for f in /proc/*/cmdline: run a for loop with loop variable f for all the args in /proc/*/cmdline echo $(\u0026lt;$f): echo the value in $f This is a nice combination of a one line for loop, globbing, command substitution and input redirection.\nChallenge 5 - Kill the rogue process! Now that we know which process is causing havoc, it is time to kill it before it does more damage!\nSolution kill -9 PID kill is a bash built-in. It passes a signal to the process. In this case, we need to pass the SIGKILL signal. Running kill -l lists all signals with their numbers. kill -NUM PID or kill -SIGNAME PID will send the signal to the process Summary All answers # List bash builtins help help # List current directory pwd # List files in pwd echo * # Read file contents echo \u0026#34;$(\u0026lt;file_name)\u0026#34; # Read cmdline of all running processes in proc for f in /proc/*/cmdline; do echo $(\u0026lt;$f); done # Kill process kill -9 PID ","permalink":"https://rrampage.github.io/2025/05/20/bash-oops-i-deleted-my-bin-dir-challenge/","summary":"\u003cp\u003e\u003ca href=\"https://oops.cmdchallenge.com\"\u003eOops I deleted my bin dir\u003c/a\u003e is a fun set of challenges to investigate a \u0026ldquo;rogue\u0026rdquo; process which has been deleting files on the system. It has deleted all programs from the \u003ccode\u003ebin\u003c/code\u003e directories like \u003ccode\u003eps\u003c/code\u003e, \u003ccode\u003ehtop\u003c/code\u003e, \u003ccode\u003elsof\u003c/code\u003e. All we have left is a shell prompt\u0026hellip;\u003c/p\u003e\n\u003ch2 id=\"bash-built-ins\"\u003eBash built-ins\u003c/h2\u003e\n\u003cp\u003eIn the challenge, the shell we are using is \u003ccode\u003ebash\u003c/code\u003e. Bash has many \u0026ldquo;built-ins\u0026rdquo; which are commands we can run without having to spawn a process and execute a binary i.e they are part of bash itself.\u003c/p\u003e\n\u003cp\u003eThe most useful one right now is \u003ccode\u003ehelp\u003c/code\u003e which lists all the bash built-ins and their arguments.\nTyping \u003ccode\u003ehelp BUILTIN_NAME\u003c/code\u003e will give more details about that built-in command.\u003c/p\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003ch3\u003eList of built-ins\u003c/h3\u003e\u003c/summary\u003e\n  \u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003ejob_spec \u003cspan class=\"o\"\u003e[\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026amp;\u003c/span\u003e\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                   \u003cspan class=\"nb\"\u003ehistory\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-c\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-d offset\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003en\u003cspan class=\"o\"\u003e]\u003c/span\u003e or \u003cspan class=\"nb\"\u003ehistory\u003c/span\u003e -anrw \u003cspan class=\"o\"\u003e[\u003c/span\u003efilename\u003cspan class=\"o\"\u003e]\u003c/span\u003e or \u003cspan class=\"nb\"\u003ehistory\u003c/span\u003e -ps\u003cspan class=\"o\"\u003e[\u003c/span\u003e1/218\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"o\"\u003e((\u003c/span\u003e expression \u003cspan class=\"o\"\u003e))\u003c/span\u003e                                                               \u003cspan class=\"k\"\u003eif\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003ethen\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e \u003cspan class=\"k\"\u003eelif\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003ethen\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"o\"\u003e]\u003c/span\u003e... \u003cspan class=\"o\"\u003e[\u003c/span\u003e \u003cspan class=\"k\"\u003eelse\u003c/span\u003e COM\u0026gt;\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e . filename \u003cspan class=\"o\"\u003e[\u003c/span\u003earguments\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                         \u003cspan class=\"nb\"\u003ejobs\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-lnprs\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ejobspec ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e or \u003cspan class=\"nb\"\u003ejobs\u003c/span\u003e -x \u003cspan class=\"nb\"\u003ecommand\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eargs\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e :                                                                              \u003cspan class=\"nb\"\u003ekill\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-s sigspec \u003cspan class=\"p\"\u003e|\u003c/span\u003e -n signum \u003cspan class=\"p\"\u003e|\u003c/span\u003e -sigspec\u003cspan class=\"o\"\u003e]\u003c/span\u003e pid \u003cspan class=\"p\"\u003e|\u003c/span\u003e jobspec ... or \u003cspan class=\"nb\"\u003ekill\u003c/span\u003e -l \u003cspan class=\"o\"\u003e[\u003c/span\u003esigs\u0026gt;\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e arg... \u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                     \u003cspan class=\"nb\"\u003elet\u003c/span\u003e arg \u003cspan class=\"o\"\u003e[\u003c/span\u003earg ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"o\"\u003e[[\u003c/span\u003e expression \u003cspan class=\"o\"\u003e]]\u003c/span\u003e                                                               \u003cspan class=\"nb\"\u003elocal\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eoption\u003cspan class=\"o\"\u003e]\u003c/span\u003e name\u003cspan class=\"o\"\u003e[=\u003c/span\u003evalue\u003cspan class=\"o\"\u003e]\u003c/span\u003e ...\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ealias\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename\u003cspan class=\"o\"\u003e[=\u003c/span\u003evalue\u003cspan class=\"o\"\u003e]\u003c/span\u003e ... \u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                 \u003cspan class=\"nb\"\u003elogout\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003en\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ebg\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ejob_spec ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                              mapfile \u003cspan class=\"o\"\u003e[\u003c/span\u003e-d delim\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-n count\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-O origin\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-s count\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-t\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-u fd\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-C callb\u0026gt;\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ebind\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-lpsvPSVX\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-m keymap\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-f filename\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-q name\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-u name\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-r keyseq\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e\u0026gt;  \u003cspan class=\"nb\"\u003epopd\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-n\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e+N \u003cspan class=\"p\"\u003e|\u003c/span\u003e -N\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ebreak\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003en\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                      \u003cspan class=\"nb\"\u003eprintf\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-v var\u003cspan class=\"o\"\u003e]\u003c/span\u003e format \u003cspan class=\"o\"\u003e[\u003c/span\u003earguments\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ebuiltin\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eshell-builtin \u003cspan class=\"o\"\u003e[\u003c/span\u003earg ...\u003cspan class=\"o\"\u003e]]\u003c/span\u003e                                              \u003cspan class=\"nb\"\u003epushd\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-n\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e+N \u003cspan class=\"p\"\u003e|\u003c/span\u003e -N \u003cspan class=\"p\"\u003e|\u003c/span\u003e dir\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ecaller\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eexpr\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                  \u003cspan class=\"nb\"\u003epwd\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-LP\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"k\"\u003ecase\u003c/span\u003e WORD in \u003cspan class=\"o\"\u003e[\u003c/span\u003ePATTERN \u003cspan class=\"o\"\u003e[\u003c/span\u003e\u003cspan class=\"p\"\u003e|\u003c/span\u003e PATTERN\u003cspan class=\"o\"\u003e]\u003c/span\u003e...\u003cspan class=\"o\"\u003e)\u003c/span\u003e COMMANDS \u003cspan class=\"p\"\u003e;;\u003c/span\u003e\u003cspan class=\"o\"\u003e]\u003c/span\u003e... \u003cspan class=\"k\"\u003eesac\u003c/span\u003e                     \u003cspan class=\"nb\"\u003eread\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-ers\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-a array\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-d delim\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-i text\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-n nchars\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-N nchars\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p pro\u0026gt;\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ecd\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-L\u003cspan class=\"p\"\u003e|\u003c/span\u003e\u003cspan class=\"o\"\u003e[\u003c/span\u003e-P \u003cspan class=\"o\"\u003e[\u003c/span\u003e-e\u003cspan class=\"o\"\u003e]]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-@\u003cspan class=\"o\"\u003e]]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003edir\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                   readarray \u003cspan class=\"o\"\u003e[\u003c/span\u003e-d delim\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-n count\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-O origin\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-s count\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-t\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-u fd\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-C cal\u0026gt;\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ecommand\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-pVv\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"nb\"\u003ecommand\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003earg ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                               \u003cspan class=\"nb\"\u003ereadonly\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-aAf\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename\u003cspan class=\"o\"\u003e[=\u003c/span\u003evalue\u003cspan class=\"o\"\u003e]\u003c/span\u003e ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e or \u003cspan class=\"nb\"\u003ereadonly\u003c/span\u003e -p\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ecompgen\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-abcdefgjksuv\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-o option\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-A action\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-G globpat\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-W wordlist\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e\u0026gt;  \u003cspan class=\"k\"\u003ereturn\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003en\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ecomplete\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-abcdefgjksuv\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-pr\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-DEI\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-o option\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-A action\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-G globpat\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e\u0026gt;  \u003cspan class=\"k\"\u003eselect\u003c/span\u003e NAME \u003cspan class=\"o\"\u003e[\u003c/span\u003ein WORDS ... \u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"k\"\u003edo\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edone\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e compopt \u003cspan class=\"o\"\u003e[\u003c/span\u003e-o\u003cspan class=\"p\"\u003e|\u003c/span\u003e+o option\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-DEI\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                       \u003cspan class=\"nb\"\u003eset\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-abefhkmnptuvxBCHP\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-o option-name\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e--\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003earg ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"k\"\u003econtinue\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003en\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                   \u003cspan class=\"nb\"\u003eshift\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003en\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e coproc \u003cspan class=\"o\"\u003e[\u003c/span\u003eNAME\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"nb\"\u003ecommand\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eredirections\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                           \u003cspan class=\"nb\"\u003eshopt\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-pqsu\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-o\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eoptname ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003edeclare\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-aAfFgiIlnrtux\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename\u003cspan class=\"o\"\u003e[=\u003c/span\u003evalue\u003cspan class=\"o\"\u003e]\u003c/span\u003e ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                               \u003cspan class=\"nb\"\u003esource\u003c/span\u003e filename \u003cspan class=\"o\"\u003e[\u003c/span\u003earguments\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003edirs\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-clpv\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e+N\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-N\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                         \u003cspan class=\"nb\"\u003esuspend\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-f\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003edisown\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-h\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-ar\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ejobspec ... \u003cspan class=\"p\"\u003e|\u003c/span\u003e pid ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                      \u003cspan class=\"nb\"\u003etest\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eexpr\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003eecho\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-neE\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003earg ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                          \u003cspan class=\"nb\"\u003etime\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p\u003cspan class=\"o\"\u003e]\u003c/span\u003e pipeline\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003eenable\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-a\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-dnps\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-f filename\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                   \u003cspan class=\"nb\"\u003etimes\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003eeval\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003earg ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                 \u003cspan class=\"nb\"\u003etrap\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-lp\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[[\u003c/span\u003earg\u003cspan class=\"o\"\u003e]\u003c/span\u003e signal_spec ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003eexec\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-cl\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-a name\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e\u003cspan class=\"nb\"\u003ecommand\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eargument ...\u003cspan class=\"o\"\u003e]]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eredirection ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                \u003cspan class=\"nb\"\u003etrue\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003eexit\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003en\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                       \u003cspan class=\"nb\"\u003etype\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-afptP\u003cspan class=\"o\"\u003e]\u003c/span\u003e name \u003cspan class=\"o\"\u003e[\u003c/span\u003ename ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003eexport\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-fn\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename\u003cspan class=\"o\"\u003e[=\u003c/span\u003evalue\u003cspan class=\"o\"\u003e]\u003c/span\u003e ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e or \u003cspan class=\"nb\"\u003eexport\u003c/span\u003e -p                                   \u003cspan class=\"nb\"\u003etypeset\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-aAfFgiIlnrtux\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p\u003cspan class=\"o\"\u003e]\u003c/span\u003e name\u003cspan class=\"o\"\u003e[=\u003c/span\u003evalue\u003cspan class=\"o\"\u003e]\u003c/span\u003e ...\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003efalse\u003c/span\u003e                                                                          \u003cspan class=\"nb\"\u003eulimit\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-SHabcdefiklmnpqrstuvxPT\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003elimit\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003efc\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-e ename\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-lnr\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003efirst\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003elast\u003cspan class=\"o\"\u003e]\u003c/span\u003e or \u003cspan class=\"nb\"\u003efc\u003c/span\u003e -s \u003cspan class=\"o\"\u003e[\u003c/span\u003e\u003cspan class=\"nv\"\u003epat\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003erep\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ecommand\u003cspan class=\"o\"\u003e]\u003c/span\u003e               \u003cspan class=\"nb\"\u003eumask\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-S\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003emode\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003efg\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ejob_spec\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                                  \u003cspan class=\"nb\"\u003eunalias\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-a\u003cspan class=\"o\"\u003e]\u003c/span\u003e name \u003cspan class=\"o\"\u003e[\u003c/span\u003ename ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"k\"\u003efor\u003c/span\u003e NAME \u003cspan class=\"o\"\u003e[\u003c/span\u003ein WORDS ... \u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edo\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edone\u003c/span\u003e                                   \u003cspan class=\"nb\"\u003eunset\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-f\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-v\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-n\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"k\"\u003efor\u003c/span\u003e \u003cspan class=\"o\"\u003e((\u003c/span\u003e exp1\u003cspan class=\"p\"\u003e;\u003c/span\u003e exp2\u003cspan class=\"p\"\u003e;\u003c/span\u003e exp3 \u003cspan class=\"o\"\u003e))\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edo\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edone\u003c/span\u003e                                  \u003cspan class=\"k\"\u003euntil\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edo\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edone\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"k\"\u003efunction\u003c/span\u003e name \u003cspan class=\"o\"\u003e{\u003c/span\u003e COMMANDS \u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"o\"\u003e}\u003c/span\u003e or name \u003cspan class=\"o\"\u003e()\u003c/span\u003e \u003cspan class=\"o\"\u003e{\u003c/span\u003e COMMANDS \u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"o\"\u003e}\u003c/span\u003e                         variables - Names and meanings of some shell variables\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003egetopts\u003c/span\u003e optstring name \u003cspan class=\"o\"\u003e[\u003c/span\u003earg ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                               \u003cspan class=\"nb\"\u003ewait\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-fn\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p var\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003eid ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ehash\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-lr\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-p pathname\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-dt\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003ename ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                      \u003cspan class=\"k\"\u003ewhile\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edo\u003c/span\u003e COMMANDS\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003edone\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e \u003cspan class=\"nb\"\u003ehelp\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e-dms\u003cspan class=\"o\"\u003e]\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003epattern ...\u003cspan class=\"o\"\u003e]\u003c/span\u003e                                                      \u003cspan class=\"o\"\u003e{\u003c/span\u003e COMMANDS \u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"o\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003c/details\u003e\n\n\u003ch2 id=\"challenge-1---print-the-current-working-directory\"\u003eChallenge 1 - print the current working directory\u003c/h2\u003e\n\u003cdetails\u003e\n  \u003csummary\u003e\n    \n    \n    \u003ch3\u003eSolution\u003c/h3\u003e\n  \u003c/summary\u003e\n  \u003cdiv\u003e\n    \nThe `pwd` built-in shows the *p*resent *w*orking *d*irectory\n\n\n  \u003c/div\u003e\n\u003c/details\u003e\n\u003ch2 id=\"challenge-2---list-all-files-in-the-current-working-directory\"\u003eChallenge 2 - List all files in the current working directory\u003c/h2\u003e\n\u003cp\u003eWe can not use \u003ccode\u003els\u003c/code\u003e as it is gone\u0026hellip; Which built-in can help us?\u003c/p\u003e","title":"Bash - Oops I deleted my bin dir challenge"},{"content":"Overview of the HTML details tag The details tag creates a toggleable element which we can use to hide sensitive information like spoilers. It has a summary attribute which is visible to the reader and a button which they can click to show then content.\n\u0026lt;details\u0026gt; \u0026lt;summary\u0026gt;Spoilers!\u0026lt;/summary\u0026gt; Lot of sensitive information! \u0026lt;/details\u0026gt; Creating Toggle Headings using summary and HTML heading tags Since summary is an HTML element, we can nest other HTML elements in it. If we want to create something like Notion\u0026rsquo;s toggle headings (/toggleh1), we can do something like\n\u0026lt;details\u0026gt; \u0026lt;summary\u0026gt;\u0026lt;h3\u0026gt;Spoilers!\u0026lt;/h3\u0026gt;\u0026lt;/summary\u0026gt; Spoilers \u0026lt;/details\u0026gt; This shows the \u0026lt;h3\u0026gt; header but below the taggle i.e not inline. See this fiddle for the minimal example. Searching for this led me to this article which suggested this CSS fix to the problem\ndetails summary \u0026gt; * { display: inline; } This CSS snippet inlines all children of the details summary element.\nHere\u0026rsquo;s the fix in action!\nUsing the details element in Hugo markdown Since version 0.140, Hugo has supported details as a shortcode. We can put arbitrary HTML in the summary e.g\n{{\u0026lt; details summary=\u0026quot;\u0026lt;h3\u0026gt;Major Spoilers!\u0026lt;/h3\u0026gt;\u0026quot; \u0026gt;}}\n{{\u0026lt; /details \u0026gt;}}\nUsing a custom Hugo Shortcodes to generate toggle headings I created a custom shortcode called tog with tag and summary attributes to support toggle headers. Here it is in action! Also linked is a gist with the minimal necessary HTML + CSS to create and use toggle headings in Hugo.\nShortcode HTML Create the following HTML in `tog.html` in the `layouts/shortcodes` directory \u0026lt;details\u003e\n\u0026lt;summary\u003e\n{{ $tag := .Get \"tag\" | default \"h3\" }}\n{{ $heading := printf \"\u003c%s\u003e%s\u003c/%s\u003e\" $tag (.Get \"summary\") $tag | safeHTML }}\n{{ $heading }}\n\u0026lt;/summary\u003e\n\u0026lt;div\u003e\n{{ .Inner }} \u0026lt;/div\u003e\n\u0026lt;/details\u003e\nUsage {{\u0026lt; tog tag=\"h4\" summary=\"Usage\" \u003e}}\n{{\u0026lt; /tog \u003e}}\n","permalink":"https://rrampage.github.io/2025/03/06/html-toggle-headings-using-details-summary-element-and-some-css/","summary":"\u003ch2 id=\"overview-of-the-html-details-tag\"\u003eOverview of the HTML \u003ccode\u003edetails\u003c/code\u003e tag\u003c/h2\u003e\n\u003cp\u003eThe \u003ca href=\"https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details\"\u003e\u003ccode\u003edetails\u003c/code\u003e tag\u003c/a\u003e creates a toggleable element which we can use to hide sensitive information like spoilers. It has a \u003ccode\u003esummary\u003c/code\u003e attribute which is visible to the reader and a button which they can click to show then content.\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-html\" data-lang=\"html\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e\u0026lt;\u003c/span\u003e\u003cspan class=\"nt\"\u003edetails\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"p\"\u003e\u0026lt;\u003c/span\u003e\u003cspan class=\"nt\"\u003esummary\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003eSpoilers!\u003cspan class=\"p\"\u003e\u0026lt;/\u003c/span\u003e\u003cspan class=\"nt\"\u003esummary\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  Lot of sensitive information!\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e\u0026lt;/\u003c/span\u003e\u003cspan class=\"nt\"\u003edetails\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch2 id=\"creating-toggle-headings-using-summary-and-html-heading-tags\"\u003eCreating Toggle Headings using \u003ccode\u003esummary\u003c/code\u003e and HTML heading tags\u003c/h2\u003e\n\u003cp\u003eSince \u003ccode\u003esummary\u003c/code\u003e is an HTML element, we can nest other HTML elements in it. If we want to create something like Notion\u0026rsquo;s toggle headings (\u003ccode\u003e/toggleh1\u003c/code\u003e), we can do something like\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-html\" data-lang=\"html\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e\u0026lt;\u003c/span\u003e\u003cspan class=\"nt\"\u003edetails\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"p\"\u003e\u0026lt;\u003c/span\u003e\u003cspan class=\"nt\"\u003esummary\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u0026lt;\u003c/span\u003e\u003cspan class=\"nt\"\u003eh3\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003eSpoilers!\u003cspan class=\"p\"\u003e\u0026lt;/\u003c/span\u003e\u003cspan class=\"nt\"\u003eh3\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u0026lt;/\u003c/span\u003e\u003cspan class=\"nt\"\u003esummary\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  Spoilers\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e\u0026lt;/\u003c/span\u003e\u003cspan class=\"nt\"\u003edetails\u003c/span\u003e\u003cspan class=\"p\"\u003e\u0026gt;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eThis shows the \u003ccode\u003e\u0026lt;h3\u0026gt;\u003c/code\u003e header but below the taggle i.e not inline. See \u003ca href=\"https://jsfiddle.net/3ht4n0jm/\"\u003ethis fiddle\u003c/a\u003e for the minimal example. Searching for this led me to this \u003ca href=\"https://css-tricks.com/two-issues-styling-the-details-element-and-how-to-solve-them/\"\u003earticle\u003c/a\u003e which suggested this CSS fix to the problem\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-css\" data-lang=\"css\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nt\"\u003edetails\u003c/span\u003e \u003cspan class=\"nt\"\u003esummary\u003c/span\u003e \u003cspan class=\"o\"\u003e\u0026gt;\u003c/span\u003e \u003cspan class=\"o\"\u003e*\u003c/span\u003e \u003cspan class=\"p\"\u003e{\u003c/span\u003e \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"k\"\u003edisplay\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"kc\"\u003einline\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eThis CSS snippet inlines all children of the \u003ccode\u003edetails summary\u003c/code\u003e element.\u003c/p\u003e\n\u003cp\u003eHere\u0026rsquo;s the \u003ca href=\"https://jsfiddle.net/0yjpcLef/\"\u003efix\u003c/a\u003e in action!\u003c/p\u003e\n\u003ch2 id=\"using-the-details-element-in-hugo-markdown\"\u003eUsing the \u003ccode\u003edetails\u003c/code\u003e element in Hugo markdown\u003c/h2\u003e\n\u003cp\u003eSince version 0.140, Hugo has supported \u003ccode\u003edetails\u003c/code\u003e as a \u003ca href=\"https://gohugo.io/shortcodes/details/\"\u003eshortcode\u003c/a\u003e. We can put arbitrary HTML in the summary e.g\u003c/p\u003e","title":"HTML toggle headings using Details Summary element and some CSS"},{"content":"Today\u0026rsquo;s problem - Caesar Salad\nA Caesar cipher, also known as a shift cipher, is one of the simplest encryption techniques. In a Caesar cipher, each letter in the plaintext is replaced by a letter some fixed number of positions away in the alphabet, effectively \u0026ldquo;shifting\u0026rdquo; the alphabet.\nWrite a function that:\ntakes a single integer left argument representing the size of the shift takes a character vector right argument representing the plaintext message returns a result with the same shape as the right argument representing the encrypted message Notes:\nUse ' ',⎕A as the alphabet You can assume that the plaintext message will contain only characters found in the alphabet. Hint: The rotate function ⌽ could be helpful in solving this problem.\nExamples 4 caesar \u0026#39;HELLO WORLDS\u0026#39; LIPPSD SVPHW ¯4 caesar \u0026#39;HELLO WORLDS\u0026#39; ⍝ negative shifts are okay DAHHKWSKNH O 0 caesar \u0026#39;HELLO WORLDS\u0026#39; ⍝ no shift is okay HELLO WORLDS 27 caesar \u0026#39;HELLO WORLDS\u0026#39; HELLO WORLDS ¯10 caesar \u0026#39;\u0026#39; ⍝ returns an empty vector Solution See the solution {(⍺⌽\u0026#39; \u0026#39;,⎕A)[(\u0026#39; \u0026#39;,⎕A) ⍸⍵]} Breaking it down:\n1. Define the alphabet: ' ',⎕A creates our alphabet consisting of a space followed by the uppercase letters A-Z. 2. Find positions in the alphabet: (' ',⎕A) ⍸⍵ uses the index-of (⍸) function to find the positions of each character in the input string (⍵) within our alphabet. 3. Create the shifted alphabet: ⍺⌽' ',⎕A rotates our alphabet by the shift amount (⍺). The rotate function (⌽) shifts the entire alphabet by the specified number of positions. 4. Map to the new characters: Using the positions found in step 2, we index into the shifted alphabet: (⍺⌽' ',⎕A)[...] to get our encrypted message. ","permalink":"https://rrampage.github.io/2025/02/24/an-apl-a-day-caesar-salad/","summary":"\u003cp\u003eToday\u0026rsquo;s problem - \u003ca href=\"https://apl.quest/psets/2023.html?goto=P3_Caesar_Salad\"\u003eCaesar Salad\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eA Caesar cipher, also known as a shift cipher, is one of the simplest encryption techniques. In a Caesar cipher, each letter in the plaintext is replaced by a letter some fixed number of positions away in the alphabet, effectively \u0026ldquo;shifting\u0026rdquo; the alphabet.\u003c/p\u003e\n\u003cp\u003eWrite a function that:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003etakes a single integer left argument representing the size of the shift\ntakes a character vector right argument representing the plaintext message\nreturns a result with the same shape as the right argument representing the encrypted message\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eNotes:\u003c/p\u003e\n\u003cpre\u003e\u003ccode\u003eUse ' ',⎕A as the alphabet\nYou can assume that the plaintext message will contain only characters found in the alphabet.\n\u003c/code\u003e\u003c/pre\u003e\n\u003cp\u003eHint: The rotate function ⌽ could be helpful in solving this problem.\u003c/p\u003e\n\u003ch2 id=\"examples\"\u003eExamples\u003c/h2\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-apl\" data-lang=\"apl\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"m\"\u003e4\u003c/span\u003e \u003cspan class=\"nv\"\u003ecaesar\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;HELLO WORLDS\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003eLIPPSD\u003c/span\u003e \u003cspan class=\"nv\"\u003eSVPHW\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"m\"\u003e¯4\u003c/span\u003e \u003cspan class=\"nv\"\u003ecaesar\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;HELLO WORLDS\u0026#39;\u003c/span\u003e \u003cspan class=\"c1\"\u003e⍝ negative shifts are okay\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003eDAHHKWSKNH\u003c/span\u003e \u003cspan class=\"nv\"\u003eO\u003c/span\u003e \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"nv\"\u003ecaesar\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;HELLO WORLDS\u0026#39;\u003c/span\u003e \u003cspan class=\"c1\"\u003e⍝ no shift is okay\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003eHELLO\u003c/span\u003e \u003cspan class=\"nv\"\u003eWORLDS\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"m\"\u003e27\u003c/span\u003e \u003cspan class=\"nv\"\u003ecaesar\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;HELLO WORLDS\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003eHELLO\u003c/span\u003e \u003cspan class=\"nv\"\u003eWORLDS\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"m\"\u003e¯10\u003c/span\u003e \u003cspan class=\"nv\"\u003ecaesar\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;\u0026#39;\u003c/span\u003e \u003cspan class=\"c1\"\u003e⍝ returns an empty vector\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch2 id=\"solution\"\u003eSolution\u003c/h2\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003ch3\u003eSee the solution\u003c/h3\u003e\u003c/summary\u003e\n  \u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-apl\" data-lang=\"apl\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kt\"\u003e{\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"bp\"\u003e⍺\u003c/span\u003e\u003cspan class=\"o\"\u003e⌽\u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39; \u0026#39;\u003c/span\u003e\u003cspan class=\"o\"\u003e,\u003c/span\u003e\u003cspan class=\"nf\"\u003e⎕A\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"sr\"\u003e[\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s1\"\u003e\u0026#39; \u0026#39;\u003c/span\u003e\u003cspan class=\"o\"\u003e,\u003c/span\u003e\u003cspan class=\"nf\"\u003e⎕A\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"o\"\u003e⍸\u003c/span\u003e\u003cspan class=\"bp\"\u003e⍵\u003c/span\u003e\u003cspan class=\"sr\"\u003e]\u003c/span\u003e\u003cspan class=\"kt\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eBreaking it down:\u003c/p\u003e","title":"An APL a day! - Caesar Salad"},{"content":"Without much ado, today\u0026rsquo;s problem - Put it in Reverse\nThe find function X⍷Y identifies the beginnings of occurrences of array X in array Y.\nIn this problem, you\u0026rsquo;re asked to return a result that identifies the endings of occurrences of array X in array Y. To keep things simple, X and Y will be at most rank 1, meaning they\u0026rsquo;ll either be vectors or scalars.\nWrite a function that:\ntakes a scalar or vector left argument takes a scalar or vector right argument returns a Boolean result that is the same shape as the right argument where 1\u0026rsquo;s mark the ends of occurrences of the left argument in the right argument Examples: \u0026#39;abra\u0026#39; findEnd \u0026#39;abracadabra\u0026#39; 0 0 0 1 0 0 0 0 0 0 1 \u0026#39;issi\u0026#39; findEnd \u0026#39;Mississippi\u0026#39; 0 0 0 0 1 0 0 1 0 0 0 \u0026#39;bb\u0026#39; findEnd \u0026#39;bbb bbb\u0026#39; 0 1 1 0 0 1 1 (,42) findEnd 42 0 42 findEnd 42 1 (,42) findEnd ,42 1 \u0026#39;are\u0026#39; \u0026#39;aquatic\u0026#39; findEnd \u0026#39;ducks\u0026#39; \u0026#39;are\u0026#39; \u0026#39;aquatic\u0026#39; \u0026#39;avians\u0026#39; 0 0 1 0 Solution See the solution {⌽(⌽⍺)⍷⌽⍵} We use ⍷ to find beginning of reversed array ⌽⍺ in ⌽⍵ We reverse this to get the ending of ⍺ in ⍵ Taking example 'abra' {⌽(⌽⍺)⍷⌽⍵} 'abracadabra'\nFor 'abra' ⍷ 'abracadabra', we get 1 0 0 0 0 0 0 1 0 0 0 as 'abra matches twice at index 1 and 8 When we reverse \u0026lsquo;abra\u0026rsquo; and \u0026lsquo;abracadabra\u0026rsquo; and do a find, we get start index of \u0026lsquo;arba\u0026rsquo; in \u0026lsquo;arbadacarba\u0026rsquo; i.e 1 0 0 0 0 0 0 1 0 0 0 Finally, when we reverse this array, we get end index of \u0026lsquo;abra\u0026rsquo; in \u0026lsquo;abracadabra\u0026rsquo; i.e 0 0 0 1 0 0 0 0 0 0 1 ","permalink":"https://rrampage.github.io/2025/02/21/an-apl-a-day-put-it-in-reverse/","summary":"\u003cp\u003eWithout much ado, today\u0026rsquo;s problem - \u003ca href=\"https://apl.quest/psets/2023.html?goto=P2_Put_It_In_Reverse\"\u003ePut it in Reverse\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eThe find function X⍷Y identifies the beginnings of occurrences of array X in array Y.\u003c/p\u003e\n\u003cp\u003eIn this problem, you\u0026rsquo;re asked to return a result that identifies the endings of occurrences of array X in array Y. To keep things simple, X and Y will be at most rank 1, meaning they\u0026rsquo;ll either be vectors or scalars.\u003c/p\u003e\n\u003cp\u003eWrite a function that:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003etakes a scalar or vector left argument\u003c/li\u003e\n\u003cli\u003etakes a scalar or vector right argument\u003c/li\u003e\n\u003cli\u003ereturns a Boolean result that is the same shape as the right argument where 1\u0026rsquo;s mark the ends of occurrences of the left argument in the right argument\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"examples\"\u003eExamples:\u003c/h2\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-apl\" data-lang=\"apl\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"s1\"\u003e\u0026#39;abra\u0026#39;\u003c/span\u003e \u003cspan class=\"nv\"\u003efindEnd\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;abracadabra\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"s1\"\u003e\u0026#39;issi\u0026#39;\u003c/span\u003e \u003cspan class=\"nv\"\u003efindEnd\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;Mississippi\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"s1\"\u003e\u0026#39;bb\u0026#39;\u003c/span\u003e \u003cspan class=\"nv\"\u003efindEnd\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;bbb bbb\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e,\u003c/span\u003e\u003cspan class=\"m\"\u003e42\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"nv\"\u003efindEnd\u003c/span\u003e \u003cspan class=\"m\"\u003e42\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e0\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"m\"\u003e42\u003c/span\u003e \u003cspan class=\"nv\"\u003efindEnd\u003c/span\u003e \u003cspan class=\"m\"\u003e42\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e1\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e,\u003c/span\u003e\u003cspan class=\"m\"\u003e42\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"nv\"\u003efindEnd\u003c/span\u003e \u003cspan class=\"o\"\u003e,\u003c/span\u003e\u003cspan class=\"m\"\u003e42\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e1\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e      \u003cspan class=\"s1\"\u003e\u0026#39;are\u0026#39;\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;aquatic\u0026#39;\u003c/span\u003e \u003cspan class=\"nv\"\u003efindEnd\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;ducks\u0026#39;\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;are\u0026#39;\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;aquatic\u0026#39;\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;avians\u0026#39;\u003c/span\u003e \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e0\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch2 id=\"solution\"\u003eSolution\u003c/h2\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eSee the solution\u003c/summary\u003e\n  \u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-apl\" data-lang=\"apl\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kt\"\u003e{\u003c/span\u003e\u003cspan class=\"o\"\u003e⌽\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"o\"\u003e⌽\u003c/span\u003e\u003cspan class=\"bp\"\u003e⍺\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"o\"\u003e⍷⌽\u003c/span\u003e\u003cspan class=\"bp\"\u003e⍵\u003c/span\u003e\u003cspan class=\"kt\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cul\u003e\n\u003cli\u003eWe use ⍷ to find beginning of reversed array \u003ccode\u003e⌽⍺\u003c/code\u003e in \u003ccode\u003e⌽⍵\u003c/code\u003e\u003c/li\u003e\n\u003cli\u003eWe reverse this to get the ending of ⍺ in ⍵\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTaking example \u003ccode\u003e'abra' {⌽(⌽⍺)⍷⌽⍵} 'abracadabra'\u003c/code\u003e\u003c/p\u003e","title":"An APL a day! - Put It In Reverse"},{"content":"I\u0026rsquo;ve started dabbling with APL. It has been a lot of fun solving the current batch of challenges. Archive. I loved how each challenge focuses on a small set of operators and makes us use them in various (devious!) combinations to solve the problems. I\u0026rsquo;m going through some of the older challenges to learn about more operators.\nThe past challenges can be found at APL Quest. Today, we\u0026rsquo;ll be solving the first problem from the 2023 challenge. Use tryapl.org as an online interactive REPL.\nElimination Sort An \u0026ldquo;Elimination Sort\u0026rdquo; is a somewhat farcical sorting algorithm which starts with the leftmost element and keeps subsequent elements that are at least as large as the previous kept element, discarding all other elements. For example:\nEliminationSort 1 3 7 3 5 8 5 8 1 6 1 8 1 10 8 4 3 4 1 4 1 3 7 8 8 8 10 Write a function that:\ntakes a non-empty numeric vector right argument returns an \u0026ldquo;Elimination-sorted\u0026rdquo; vector of the right argument Hint: The progressive-maxima idiomatic phrase ⌈, the greater or equal function ≥, and the replicate function / could be helpful in solving this problem.\nExamples EliminationSort ⍳10 1 2 3 4 5 6 7 8 9 10 EliminationSort 2 1 4 3 6 5 8 7 10 9 2 4 6 8 10 EliminationSort 1000 2500 1333 1969 3141 2345 3141 4291.9 4291.8 4292 1000 2500 3141 3141 4291.9 4292 EliminationSort 1 3 7 3 5 8 5 8 1 6 1 8 1 10 8 4 3 4 1 4 1 3 7 8 8 8 10 Solution See the solution {(⍵=⌈\\⍵)/⍵} ⌈\\⍵ to generate a scan of running maximum e.g for ⌈\\2 1 4 3 6 5 8 7 10 9 gives 2 2 4 4 6 6 8 8 10 10 Compare with ⍵ to only get those indices where the actual element exists {(⍵=⌈\\⍵)} 2 1 4 3 6 5 8 7 10 9 gives 1 0 1 0 1 0 1 0 1 0 Finally, replicate (/) to get only the required elements {(⍵=⌈\\⍵)/⍵} 2 1 4 3 6 5 8 7 10 9 gives 2 4 6 8 10 ","permalink":"https://rrampage.github.io/2025/02/20/an-apl-a-day-elimination-sort/","summary":"\u003cp\u003eI\u0026rsquo;ve started dabbling with APL. It has been a lot of fun solving the \u003ca href=\"https://challenge.dyalog.com/\"\u003ecurrent batch of challenges\u003c/a\u003e. \u003ca href=\"https://web.archive.org/web/20250210194607/https://challenge.dyalog.com/?goto=welcome\"\u003eArchive\u003c/a\u003e. I loved how each challenge focuses on a small set of operators and makes us use them in various (devious!) combinations to solve the problems. I\u0026rsquo;m going through some of the older challenges to learn about more operators.\u003c/p\u003e\n\u003cp\u003eThe past challenges can be found at \u003ca href=\"https://apl.quest/\"\u003eAPL Quest\u003c/a\u003e. Today, we\u0026rsquo;ll be solving the first problem from the 2023 challenge. Use \u003ca href=\"https://tryapl.org/\"\u003etryapl.org\u003c/a\u003e as an online interactive REPL.\u003c/p\u003e\n\u003ch1 id=\"elimination-sort\"\u003e\u003ca href=\"https://apl.quest/psets/2023.html?goto=P1_Elimination_Sort\"\u003eElimination Sort\u003c/a\u003e\u003c/h1\u003e\n\u003cp\u003eAn \u0026ldquo;Elimination Sort\u0026rdquo; is a somewhat farcical sorting algorithm which starts with the leftmost element and keeps subsequent elements that are at least as large as the previous kept element, discarding all other elements. For example:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-apl\" data-lang=\"apl\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003eEliminationSort\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e3\u003c/span\u003e \u003cspan class=\"m\"\u003e7\u003c/span\u003e \u003cspan class=\"m\"\u003e3\u003c/span\u003e \u003cspan class=\"m\"\u003e5\u003c/span\u003e \u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e5\u003c/span\u003e \u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e6\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e10\u003c/span\u003e \u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e4\u003c/span\u003e \u003cspan class=\"m\"\u003e3\u003c/span\u003e \u003cspan class=\"m\"\u003e4\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e4\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e1\u003c/span\u003e \u003cspan class=\"m\"\u003e3\u003c/span\u003e \u003cspan class=\"m\"\u003e7\u003c/span\u003e \u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e10\u003c/span\u003e \n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eWrite a function that:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003etakes a non-empty numeric vector right argument\u003c/li\u003e\n\u003cli\u003ereturns an \u0026ldquo;Elimination-sorted\u0026rdquo; vector of the right argument\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eHint: The progressive-maxima idiomatic phrase ⌈, the greater or equal function ≥, and the replicate function / could be helpful in solving this problem.\u003c/p\u003e","title":"An APL a day! - Elimination Sort"},{"content":"Lua - Basics for x = 0, 10, 1 do print(x) end -- for loop with range and step. step if not given is assumed to be 1 -- Tables are the soul of Lua. They double up as arrays and hashes a = {} b = {1, 2, 3} b[1] -- b[1] will return 1 as indexing for arrays is from 1 c = {[0]={123,3},[\u0026#34;a\u0026#34;]={1,2,3},v={2,6,7}} -- We can have a \u0026#34;0\u0026#34;th index by adding the 1st element as [0]=value -- Adding keys which are not valid identifiers in Lua needs to be done by wrapping key in [] -- If a key is a valid identifier, we can access using `.` e.g c.a for above code vs c[0] for k,v in pairs(c) do print(type(k)) end -- iterates through k/v in table and prints type of key Print a table as valid Lua code function printTable(tbl) -- Specify function vars as local else they become global local s = \u0026#34;{\u0026#34; local fst = true for k,v in pairs(tbl) do local tk = type(k) local tv = type(v) if not fst then s = s .. \u0026#34;,\u0026#34; else fst = false end if tk == \u0026#34;number\u0026#34; or tk == \u0026#34;boolean\u0026#34; then s = s .. \u0026#34;[\u0026#34; .. tostring(k) .. \u0026#34;]=\u0026#34; else s = s .. \u0026#39;[\u0026#34;\u0026#39; .. k .. \u0026#39;\u0026#34;]=\u0026#39; end if tv == \u0026#34;table\u0026#34; then s = s .. printTable(v) elseif tv == \u0026#34;number\u0026#34; or tv == \u0026#34;boolean\u0026#34; then s = s .. tostring(v) elseif tv == \u0026#34;string\u0026#34; then s = s .. \u0026#39;\u0026#34;\u0026#39; .. v .. \u0026#39;\u0026#34;\u0026#39; else s = s .. vt end end s = s .. \u0026#34;}\u0026#34; return s end sv = printTable({[0]={123,3},[\u0026#34;a\u0026#34;]={1,2,3},v={2,6,7},[true]=123}) print(sv) -- posts.lua (saved table) can be read from another lua file as posts=require(\u0026#34;posts\u0026#34;) return {[0]={[1]=123,[2]=3},[\u0026#34;v\u0026#34;]={[1]=2,[2]=6,[3]=7},[\u0026#34;a\u0026#34;]={[1]=1,[2]=2,[3]=3},[true]=123} ","permalink":"https://rrampage.github.io/2021/12/04/relearning-lua/","summary":"\u003ch2 id=\"lua---basics\"\u003eLua - Basics\u003c/h2\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-lua\" data-lang=\"lua\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kr\"\u003efor\u003c/span\u003e \u003cspan class=\"n\"\u003ex\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"mi\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"mi\"\u003e10\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"mi\"\u003e1\u003c/span\u003e \u003cspan class=\"kr\"\u003edo\u003c/span\u003e \u003cspan class=\"n\"\u003eprint\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ex\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"kr\"\u003eend\u003c/span\u003e \u003cspan class=\"c1\"\u003e-- for loop with range and step. step if not given is assumed to be 1\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- Tables are the soul of Lua. They double up as arrays and hashes\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003ea\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"p\"\u003e{}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eb\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eb\u003c/span\u003e\u003cspan class=\"p\"\u003e[\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- b[1] will return 1 as indexing for arrays is from 1\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003ec\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"p\"\u003e{[\u003c/span\u003e\u003cspan class=\"mi\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"mi\"\u003e123\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e},[\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;a\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e},\u003c/span\u003e\u003cspan class=\"n\"\u003ev\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e6\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e7\u003c/span\u003e\u003cspan class=\"p\"\u003e}}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- We can have a \u0026#34;0\u0026#34;th index by adding the 1st element as [0]=value\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- Adding keys which are not valid identifiers in Lua needs to be done by wrapping key in []\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- If a key is a valid identifier, we can access using `.` e.g c.a for above code vs c[0]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kr\"\u003efor\u003c/span\u003e \u003cspan class=\"n\"\u003ek\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"n\"\u003ev\u003c/span\u003e \u003cspan class=\"kr\"\u003ein\u003c/span\u003e \u003cspan class=\"n\"\u003epairs\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ec\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"kr\"\u003edo\u003c/span\u003e \u003cspan class=\"n\"\u003eprint\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003etype\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ek\u003c/span\u003e\u003cspan class=\"p\"\u003e))\u003c/span\u003e \u003cspan class=\"kr\"\u003eend\u003c/span\u003e \u003cspan class=\"c1\"\u003e-- iterates through k/v in table and prints type of key\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch3 id=\"print-a-table-as-valid-lua-code\"\u003ePrint a table as valid Lua code\u003c/h3\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-lua\" data-lang=\"lua\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kr\"\u003efunction\u003c/span\u003e \u003cspan class=\"nf\"\u003eprintTable\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003etbl\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\t\t\u003cspan class=\"c1\"\u003e-- Specify function vars as local else they become global\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"kd\"\u003elocal\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;{\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"kd\"\u003elocal\u003c/span\u003e \u003cspan class=\"n\"\u003efst\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"kc\"\u003etrue\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"kr\"\u003efor\u003c/span\u003e \u003cspan class=\"n\"\u003ek\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"n\"\u003ev\u003c/span\u003e \u003cspan class=\"kr\"\u003ein\u003c/span\u003e \u003cspan class=\"n\"\u003epairs\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003etbl\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"kr\"\u003edo\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \t    \u003cspan class=\"kd\"\u003elocal\u003c/span\u003e \u003cspan class=\"n\"\u003etk\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003etype\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ek\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \t    \u003cspan class=\"kd\"\u003elocal\u003c/span\u003e \u003cspan class=\"n\"\u003etv\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003etype\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ev\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eif\u003c/span\u003e \u003cspan class=\"ow\"\u003enot\u003c/span\u003e \u003cspan class=\"n\"\u003efst\u003c/span\u003e \u003cspan class=\"kr\"\u003ethen\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                        \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;,\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eelse\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                        \u003cspan class=\"n\"\u003efst\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"kc\"\u003efalse\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eend\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eif\u003c/span\u003e \u003cspan class=\"n\"\u003etk\u003c/span\u003e \u003cspan class=\"o\"\u003e==\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;number\u0026#34;\u003c/span\u003e \u003cspan class=\"ow\"\u003eor\u003c/span\u003e \u003cspan class=\"n\"\u003etk\u003c/span\u003e \u003cspan class=\"o\"\u003e==\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;boolean\u0026#34;\u003c/span\u003e \u003cspan class=\"kr\"\u003ethen\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                        \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;[\u0026#34;\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"n\"\u003etostring\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ek\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;]=\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eelse\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                        \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;[\u0026#34;\u0026#39;\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"n\"\u003ek\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;\u0026#34;]=\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eend\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eif\u003c/span\u003e \u003cspan class=\"n\"\u003etv\u003c/span\u003e \u003cspan class=\"o\"\u003e==\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;table\u0026#34;\u003c/span\u003e \u003cspan class=\"kr\"\u003ethen\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                    \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"n\"\u003eprintTable\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ev\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eelseif\u003c/span\u003e \u003cspan class=\"n\"\u003etv\u003c/span\u003e \u003cspan class=\"o\"\u003e==\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;number\u0026#34;\u003c/span\u003e \u003cspan class=\"ow\"\u003eor\u003c/span\u003e \u003cspan class=\"n\"\u003etv\u003c/span\u003e \u003cspan class=\"o\"\u003e==\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;boolean\u0026#34;\u003c/span\u003e \u003cspan class=\"kr\"\u003ethen\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                    \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"n\"\u003etostring\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ev\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eelseif\u003c/span\u003e \u003cspan class=\"n\"\u003etv\u003c/span\u003e \u003cspan class=\"o\"\u003e==\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;string\u0026#34;\u003c/span\u003e \u003cspan class=\"kr\"\u003ethen\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                    \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;\u0026#34;\u0026#39;\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"n\"\u003ev\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;\u0026#34;\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eelse\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \t\u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"n\"\u003evt\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"kr\"\u003eend\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"kr\"\u003eend\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e \u003cspan class=\"o\"\u003e..\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;}\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"kr\"\u003ereturn\u003c/span\u003e \u003cspan class=\"n\"\u003es\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kr\"\u003eend\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003esv\u003c/span\u003e \u003cspan class=\"o\"\u003e=\u003c/span\u003e \u003cspan class=\"n\"\u003eprintTable\u003c/span\u003e\u003cspan class=\"p\"\u003e({[\u003c/span\u003e\u003cspan class=\"mi\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"mi\"\u003e123\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e},[\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;a\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e},\u003c/span\u003e\u003cspan class=\"n\"\u003ev\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e6\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"mi\"\u003e7\u003c/span\u003e\u003cspan class=\"p\"\u003e},[\u003c/span\u003e\u003cspan class=\"kc\"\u003etrue\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e123\u003c/span\u003e\u003cspan class=\"p\"\u003e})\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eprint\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003esv\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-lua\" data-lang=\"lua\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e-- posts.lua (saved table) can be read from another lua file as posts=require(\u0026#34;posts\u0026#34;)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kr\"\u003ereturn\u003c/span\u003e \u003cspan class=\"p\"\u003e{[\u003c/span\u003e\u003cspan class=\"mi\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{[\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e123\u003c/span\u003e\u003cspan class=\"p\"\u003e,[\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e},[\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;v\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{[\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,[\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e6\u003c/span\u003e\u003cspan class=\"p\"\u003e,[\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e7\u003c/span\u003e\u003cspan class=\"p\"\u003e},[\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;a\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"p\"\u003e{[\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e,[\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,[\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e},[\u003c/span\u003e\u003cspan class=\"kc\"\u003etrue\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"mi\"\u003e123\u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e","title":"Re-learning Lua"},{"content":"Introduction The best way to learn a skill is through deliberate practice! S3 Game is an awesome game made by Vasily Pantyukhin which walks you through various features of S3 from beginner to fairly advanced.\nWhat you will need: A browser aws cli CLICK HERE TO PLAY! Each level consists of finding a \u0026ldquo;treasure\u0026rdquo; object and getting to the next level using the secret code in the \u0026ldquo;treasure\u0026rdquo;.\nGo ahead and play this game. Each level contains multiple hints with links to S3 API documentation.\nBelow is a walk through you can refer to if you are stuck on any level. Make sure that you try for a bit before seeing here :)\nWalk through (Spoilers) See the solution Level 1 Solution: # List the objects in bucket aws s3 ls s3://s3game-level1 Visit the treasure object\nLevel 2 Solution: Just get the treasure2 object either using browser or aws s3 : https://s3game-level2.s3.us-east-2.amazonaws.com/treasure2 Make sure to save the Access and Secret key somewhere\nLevel 3 Create an AWS profile \u0026ldquo;tmp\u0026rdquo; with the access and secret key obtained from previous level. You will need to use this for almost every level from here on.\naws configure --profile tmp \u0026gt; AWS Access Key ID \u0026gt; AWS Secret Access Key \u0026gt; Default region name - us-east-2 Solution: aws s3 ls s3://s3game-level3 --profile tmp aws s3 cp s3://s3game-level3/treasure3_has_no_secret_code . --profile tmp cat treasure3_has_no_secret_code # The object data itself does not have the secret code. It is present in the metadata aws s3api head-object --bucket s3game-level3 --key treasure3_has_no_secret_code --profile tmp Level 4 Hint Solution: # List the bucket .. # get the file aws s3 cp s3://s3game-level4-k73045aztqln/treasure4_also_has_no_secret_code . --profile tmp # looks like the data is empty aws s3api head-object --bucket s3game-level4-k73045aztqln --profile tmp --key treasure4_also_has_no_secret_code # ... so is the metadata. Let\u0026#39;s try object tags as mentioned in hint aws s3api get-object-tagging --bucket s3game-level4-k73045aztqln --profile tmp --key treasure4_also_has_no_secret_code Takeaway: Object in S3, including its metadata, is immutable. When you edit object\u0026rsquo;s metadata, you are actually overwriting the object with a copy of itself, with its metadata modified. In contrast, tags are subresources. They are managed separately and can be modified without modifying the object itself.\nLevel 5 Hint Hint Hint Solution # We are listing all versions of objects as the hint says that the object is not present in its current version aws s3api list-object-versions --bucket s3game-level5-8v95e5rv7z4i --profile tmp # Get the specific version of treasure5_is_deleted using version-id from previous call aws s3api get-object --bucket \u0026#39;s3game-level5-8v95e5rv7z4i\u0026#39; --key treasure5_is_deleted --version-id \u0026#39;344PQOyFqocF0TI66MbLynNNdQqHfBz3\u0026#39; --profile tmp treasure5_is_deleted Takeaway S3 versioning keeps multiple variants of an object in the same bucket. If you enable versioning for a bucket, S3 automatically generates a unique version ID for the object being stored. In one bucket you can have two objects with the same key, but different version IDs. Check if versioning is enabled using: aws s3api get-bucket-versioning --bucket my-bucket\nLevel 6 Hint Solution # We need to get a gzipped object using S3 select aws s3api list-objects --bucket s3game-level6-vjv45x1gux81 --profile tmp # Now, do the select on s3select.csv.gz aws s3api select-object-content --bucket s3game-level6-vjv45x1gux81 --key s3select.csv.gz --expression \u0026#34;SELECT s.Answer FROM s3object s WHERE Category = \u0026#39;TREASURE\u0026#39;\u0026#34; --expression-type sql --input-serialization \u0026#39;{\u0026#34;CSV\u0026#34;: {\u0026#34;FileHeaderInfo\u0026#34;: \u0026#34;USE\u0026#34;, \u0026#34;FieldDelimiter\u0026#34;: \u0026#34;;\u0026#34;}, \u0026#34;CompressionType\u0026#34;: \u0026#34;GZIP\u0026#34;}\u0026#39; --output-serialization \u0026#39;{\u0026#34;CSV\u0026#34;: {}}\u0026#39; --profile tmp treasure6 Level 7 Hint Solution # Get \u0026#39;somethingstrange\u0026#39; from the bucket aws s3api get-object --bucket s3game-level7-zhovpo4j8588 --key somethingstrange --profile tmp t7 cat t7 # Visit the pre-signed URL in the file to get URL for next level Takeaway Pre-signed URLs are a powerful feature of S3. From the docs:\nA presigned URL is generated by an AWS user who has access to the object. The generated URL is then given to the unauthorized user. The presigned URL can be entered in a browser or used by a program or HTML webpage. The credentials used by the presigned URL are those of the AWS user who generated the URL.\nA presigned URL remains valid for a limited period of time which is specified when the URL is generated.\nLevel 8 Solution # List objects aws s3api list-objects --bucket s3game-level8-v6g8tp7ra2ld --profile tmp # Visit the treasure file using the cloudfront URL curl \u0026#39;https://d2suiw06vujwz3.cloudfront.net/treasure8_CDN\u0026#39; Takeaway You can easily create a Cloudfront distribution backed by an S3 bucket In such cases, you can restrict public access for the objects in the bucket as they can be accessed using Cloudfront Ensure that your bucket does not allow unprivileged users to list all objects in the bucket Level 9 Hint The bucket has a policy which checks that any request for \u0026ldquo;arn:aws:s3:::s3game-level9-781xtls2quvy/treasure9_referer\u0026rdquo; has the referer string \u0026ldquo;http://s3game.treasure\u0026rdquo;\nSolution All we need to do is curl with a \u0026ndash;refer given in policy curl \u0026lsquo;https://s3game-level9-781xtls2quvy.s3.us-east-2.amazonaws.com/treasure9_referer' \u0026ndash;refer \u0026lsquo;http://s3game.treasure\u0026rsquo;\nLevel 10 Here, we need to fetch an object which is in Infrequent Access storage.\nSolution # We can use the powerful query param of aws cli to just fetch those objects which are in infrequent access storgae aws s3api list-objects --bucket s3game-level10-gac6tf83erp6 --query \u0026#39;Contents[?StorageClass == `STANDARD_IA`]\u0026#39; --profile tmp Takeaway Amazon S3 offers a range of storage classes designed for different use cases:\nStandard for general-purpose storage of frequently accessed data Intelligent-Tiering for data with unknown or changing access patterns Standard-Infrequent Access (Standard-IA) and One Zone-Infrequent Access (One Zone-IA) for long-lived, but less frequently accessed data Glacier and Glacier Deep Archive for long-term archive Level 11 Here we need to fetch an object which has been encrypted using client side encryption and uploaded to S3. This means that we can not fetch the object unless we have the encryption key.\nSolution # The encryption key is given in the hint aws s3 cp --sse-c \u0026#39;AES256\u0026#39; --sse-c-key \u0026#39;UkXp2s5v8y/B?E(H+MbPeShVmYq3t6w9\u0026#39; \u0026#39;s3://s3game-level11-djq30a807iyq/treasure11_encryption\u0026#39; --profile tmp . # Alternate way using aws s3api. Here we also need to give md5 of encryption key as additional integrity check aws s3api get-object --bucket s3game-level11-djq30a807iyq --key treasure11_encryption --sse-customer-key \u0026#39;UkXp2s5v8y/B?E(H+MbPeShVmYq3t6w9\u0026#39; --sse-customer-algorithm AES256 --profile tmp treasure11_encryption Takeaway S3 supports both server side and client side encryption.\nVICTORY! - Level 12 Congrats! You have cleared the game!\n","permalink":"https://rrampage.github.io/2020/06/04/s3-game/","summary":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eThe best way to learn a skill is through deliberate practice!\n\u003ca href=\"http://s3game-level1.s3-website.us-east-2.amazonaws.com/\"\u003eS3 Game\u003c/a\u003e is an awesome game made by \u003ca href=\"https://www.linkedin.com/in/vpantyukhin/\"\u003eVasily Pantyukhin\u003c/a\u003e which walks you through various features of S3 from beginner to fairly advanced.\u003c/p\u003e\n\u003ch3 id=\"what-you-will-need\"\u003eWhat you will need:\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eA browser\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html\"\u003eaws cli\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"click\"\u003e\u003cstrong\u003eCLICK \u003ca href=\"http://s3game-level1.s3-website.us-east-2.amazonaws.com/\"\u003eHERE\u003c/a\u003e TO PLAY!\u003c/strong\u003e\u003c/h3\u003e\n\u003cp\u003eEach level consists of finding a \u0026ldquo;treasure\u0026rdquo; object and getting to the next level using the secret code in the \u0026ldquo;treasure\u0026rdquo;.\u003c/p\u003e\n\u003cp\u003eGo ahead and play this game. Each level contains multiple hints with links to S3 API documentation.\u003c/p\u003e\n\u003cp\u003eBelow is a walk through you can refer to if you are stuck on any level. Make sure that you try for a bit before seeing here :)\u003c/p\u003e\n\u003ch2 id=\"walk-through-spoilers\"\u003eWalk through (Spoilers)\u003c/h2\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eSee the solution\u003c/summary\u003e\n  \u003ch3 id=\"level-1\"\u003e\u003ca href=\"http://s3game-level1.s3-website.us-east-2.amazonaws.com/level1.html\"\u003eLevel 1\u003c/a\u003e\u003c/h3\u003e\n\u003ch4 id=\"solution\"\u003eSolution:\u003c/h4\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# List the objects in bucket\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003eaws s3 ls s3://s3game-level1\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eVisit the \u003ca href=\"http://s3game-level1.s3-website.us-east-2.amazonaws.com/treasure1\"\u003etreasure object\u003c/a\u003e\u003c/p\u003e\n\u003ch3 id=\"level-2\"\u003e\u003ca href=\"https://s3game-level2.s3.us-east-2.amazonaws.com/level2-748l6b6xwzl6.html\"\u003eLevel 2\u003c/a\u003e\u003c/h3\u003e\n\u003ch4 id=\"solution-1\"\u003eSolution:\u003c/h4\u003e\n\u003cp\u003eJust get the treasure2 object either using browser or aws s3 : \u003ca href=\"https://s3game-level2.s3.us-east-2.amazonaws.com/treasure2\"\u003ehttps://s3game-level2.s3.us-east-2.amazonaws.com/treasure2\u003c/a\u003e\nMake sure to save the Access and Secret key somewhere\u003c/p\u003e\n\u003ch3 id=\"level-3\"\u003e\u003ca href=\"https://s3game-level3.s3.us-east-2.amazonaws.com/level3-76qp7mlpzyg1.html\"\u003eLevel 3\u003c/a\u003e\u003c/h3\u003e\n\u003cp\u003eCreate an AWS profile \u0026ldquo;tmp\u0026rdquo; with the access and secret key obtained from previous level. You will need to use this for almost every level from here on.\u003c/p\u003e","title":"S3 Game"},{"content":"NPM is used as a convenient cross-platform package manager for a lot of developer tools. For many tools, the defacto way to install is npm install -g $TOOL. But installing anything via npm allows it to run untrusted code on your machine.\nHere are a few tips to minimize the risk:\n1. NEVER run npm as sudo/root Node\u0026rsquo;s official documentation recommends not installing global packages as sudo/root. If you have already installed node through nvm ignore this step. If you use a system installed node e.g using Ubuntu\u0026rsquo;s apt-get, read through this guide for Linux/Mac or npm-g-nosudo which is a shell script for Linux.\nIf you are lazy (like me), here\u0026rsquo;s a summary from the linked guide: mkdir -p \u0026#34;${HOME}/.npm-packages\u0026#34; \u0026amp;\u0026amp; npm config set prefix \u0026#34;${HOME}/.npm-packages\u0026#34; Add this to your .bashrc or .zshrc:\nNPM_PACKAGES=\u0026#34;${HOME}/.npm-packages\u0026#34; export PATH=\u0026#34;$PATH:$NPM_PACKAGES/bin\u0026#34; # Preserve MANPATH if you already defined it somewhere in your config. # Otherwise, fall back to `manpath` so we can inherit from `/etc/manpath`. export MANPATH=\u0026#34;${MANPATH-$(manpath)}:$NPM_PACKAGES/share/man\u0026#34; 2. Install/Use node using nvm or asdf Node\u0026rsquo;s official documentation recommends installing node as an unprivileged user using a node version manager.\nHere are the steps:\nInstall nvm. This does not work for Windows, use nvm-windows which is a totally different project from nvm. Unfortunately, nvm suffers from the curl pipe to bash install as its main way to install. If you have git installed on your machine, you can also directly clone it using the following steps: cd \u0026quot;$HOME\u0026quot; \u0026amp;\u0026amp; git clone https://github.com/nvm-sh/nvm.git .nvm cd \u0026quot;$HOME/.nvm\u0026quot; \u0026amp;\u0026amp; git checkout v0.35.3 \u0026amp;\u0026amp; . nvm.sh Add to your .bashrc or .zshrc: export NVM_DIR=\u0026#34;$HOME/.nvm\u0026#34; [ -s \u0026#34;$NVM_DIR/nvm.sh\u0026#34; ] \u0026amp;\u0026amp; \\. \u0026#34;$NVM_DIR/nvm.sh\u0026#34; # This loads nvm [ -s \u0026#34;$NVM_DIR/bash_completion\u0026#34; ] \u0026amp;\u0026amp; \\. \u0026#34;$NVM_DIR/bash_completion\u0026#34; # This loads nvm bash_completion nvm install --lts or whichever version of node you need. nvm use node To use a node version by default, echo \u0026quot;lts/*\u0026quot; \u0026gt; \u0026quot;$HOME/.nvmrc\u0026quot; 3. List all your globally installed npm packages and remove any unnecessary ones Listing global packages is done with npm ls -g --depth 0 Remove any unnecessary packages with npm uninstall -g $TOOL You can check your shell history/scripts folders to see if you actually use many of the global packages.\n4. Audit your packages Run npm audit in your project regularly to see if any dependencies are vulnerable. This only works for your development projects, not global packages.\nA (hacky) way to npm audit global packages: Run npm. The last line will show the folder of the global npm packages e.g $HOME/.npm-packages/lib/node_modules/npm Go to $HOME/.npm-packages/lib and run npm init -y and then run npm i --package-lock-only. These steps are required as npm audit checks for package.json and package-lock.json Now run npm audit. Remove any dependencies which have high or too many moderate vulnerabilities. These could either be malicious or unmaintained tools. Remove the package.json and package-lock.json after the audit 5. Use npx to run executables Many times, it is not necessary to have a tool installed globally e.g create-react-app. Node 6+ comes with npx which allows you to temporarily download and run scripts. The benefits of using npx over npm install -g are beautifully explained in this post.\nA gist of the article:\nCalling npx when isn’t already in your $PATH will automatically install a package with that name from the npm registry for you, and invoke it. When it’s done, the installed package won’t be anywhere in your globals, so you won’t have to worry about pollution in the long-term.\nThis feature is ideal for things like generators, too. Tools like yeoman or create-react-app only ever get called once in a blue moon. By the time you run them again, they’ll already be far out of date, so you end up having to run an install every time you want to use them anyway.\nBonus : Use Deno for developer tools I think a project like deno with good sandbox capabilities and limited permisisons is the best choice for a lot of developer tools currently made using Node. Read this post for more details on why everyone is talking about Deno.\nConclusion Does this solve all security issues of node/npm on your machine? No! There are a lot of ways in which a malicious attacker can compromise your machine/project. This only prevents a bad npm tool from getting root access and removes globally installed npm tools with known insecure dependencies.\n","permalink":"https://rrampage.github.io/2020/05/28/npm-and-node-tips-to-make-your-dev-machine-slightly-safer/","summary":"\u003cp\u003eNPM is used as a convenient cross-platform package manager for \u003ca href=\"https://hackernoon.com/the-best-of-npm-install-g-9ab9d749eeb1\"\u003ea lot of developer tools\u003c/a\u003e. For many tools, the defacto way to install is \u003cstrong\u003e\u003ccode\u003enpm install -g $TOOL\u003c/code\u003e\u003c/strong\u003e. But installing anything via npm allows it to run untrusted code on your machine.\u003c/p\u003e\n\u003cp\u003eHere are a few tips to minimize the risk:\u003c/p\u003e\n\u003ch2 id=\"1-never-run-npm-as-sudoroot\"\u003e1. NEVER run npm as sudo/root\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally\"\u003eNode\u0026rsquo;s official documentation\u003c/a\u003e recommends not installing global packages as sudo/root. If you have already installed node through \u003ccode\u003envm\u003c/code\u003e ignore this step. If you use a system installed \u003ccode\u003enode\u003c/code\u003e e.g using Ubuntu\u0026rsquo;s apt-get, read through \u003cstrong\u003e\u003ca href=\"https://github.com/sindresorhus/guides/blob/master/npm-global-without-sudo.md\"\u003ethis guide\u003c/a\u003e\u003c/strong\u003e for Linux/Mac or \u003ca href=\"https://github.com/glenpike/npm-g_nosudo\"\u003enpm-g-nosudo\u003c/a\u003e which is a shell script for Linux.\u003c/p\u003e\n\u003ch3 id=\"if-you-are-lazy-like-me-heres-a-summary-from-the-linked-guide\"\u003eIf you are lazy (like me), here\u0026rsquo;s a summary from the linked guide:\u003c/h3\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003emkdir -p \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"si\"\u003e${\u003c/span\u003e\u003cspan class=\"nv\"\u003eHOME\u003c/span\u003e\u003cspan class=\"si\"\u003e}\u003c/span\u003e\u003cspan class=\"s2\"\u003e/.npm-packages\u0026#34;\u003c/span\u003e \u003cspan class=\"o\"\u003e\u0026amp;\u0026amp;\u003c/span\u003e npm config \u003cspan class=\"nb\"\u003eset\u003c/span\u003e prefix \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"si\"\u003e${\u003c/span\u003e\u003cspan class=\"nv\"\u003eHOME\u003c/span\u003e\u003cspan class=\"si\"\u003e}\u003c/span\u003e\u003cspan class=\"s2\"\u003e/.npm-packages\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eAdd this to your \u003ccode\u003e.bashrc\u003c/code\u003e or \u003ccode\u003e.zshrc\u003c/code\u003e:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eNPM_PACKAGES=\u0026#34;${HOME}/.npm-packages\u0026#34;\nexport PATH=\u0026#34;$PATH:$NPM_PACKAGES/bin\u0026#34;\n# Preserve MANPATH if you already defined it somewhere in your config.\n# Otherwise, fall back to `manpath` so we can inherit from `/etc/manpath`.\nexport MANPATH=\u0026#34;${MANPATH-$(manpath)}:$NPM_PACKAGES/share/man\u0026#34;\n\u003c/code\u003e\u003c/pre\u003e\u003ch2 id=\"2-installuse-node-using-nvm-or-asdf\"\u003e2. Install/Use node using \u003ccode\u003envm\u003c/code\u003e or \u003ccode\u003easdf\u003c/code\u003e\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally\"\u003eNode\u0026rsquo;s official documentation\u003c/a\u003e recommends installing node as an unprivileged user using a node version manager.\u003c/p\u003e","title":"NPM and Node tips to make your dev machine (slightly) safer"},{"content":"What is a webhook? A webhook is an endpoint on your server which allows you to execute a particular task. Webhooks are usually triggered by some event. A good use-case for a webhook is running tests on a dedicated test server or deploying your latest master branch to staging/production.\nGithub / Gitlab / Bitbucket allow you to specify a webhook URL in your repository settings. Github triggers the webhook which sends the event data on every push.\nWebhook server Webhook is a very useful golang project which runs any script you specify when a particular endpoint is hit.\nDownload and extract the binary for your operating system from the releases page. For Linux, it is here.\nThe program takes as config a hooks.json file:\n[ { \u0026#34;id\u0026#34;: \u0026#34;hello-world\u0026#34;, \u0026#34;execute-command\u0026#34;: \u0026#34;/home/user/scripts/hello.sh\u0026#34;, \u0026#34;command-working-directory\u0026#34;: \u0026#34;/home/user/webhook\u0026#34; } ] Replace user with the username of your linux user.\nThe hello.sh script.\n#!/bin/bash echo \u0026#39;Hello!\u0026#39; Make the script executable by running chmod +x hello.sh\nStart webhook server as webhook -hooks hooks.json -hotreload -logfile webhooks.log. The server will run on port 9000 by default. You can check if everything is working by running curl http://localhost:9000/hooks/hello-world. This will print \u0026ldquo;Hello!\u0026rdquo; in the log file.\nDeploy script For the purpose of this post, I\u0026rsquo;ll assume the script is called deploy and is at location /home/user/scripts/deploy. This script will vary depending on your tech stack and the complexity of your CI process.\nA simple example deploy script:\n#!/bin/bash # If you have a build server which creates binary/jar/artifact wget \u0026#39;ARTIFACT_URL\u0026#39; # Else, git pull and build on the server itself # Assuming this script stops old instance of your code and starts a new instance with latest artifact restart-service.sh Configuration to run deploy script [ { \u0026#34;id\u0026#34;: \u0026#34;deploy-from-git\u0026#34;, \u0026#34;execute-command\u0026#34;: \u0026#34;/home/user/scripts/deploy\u0026#34;, \u0026#34;command-working-directory\u0026#34;: \u0026#34;/home/user/scripts\u0026#34;, \u0026#34;trigger-rule\u0026#34;: { \u0026#34;and\u0026#34;: [ { \u0026#34;match\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;payload-hash-sha1\u0026#34;, \u0026#34;secret\u0026#34;: \u0026#34;MyTotallySecretString\u0026#34;, \u0026#34;parameter\u0026#34;: { \u0026#34;source\u0026#34;: \u0026#34;header\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;X-Hub-Signature\u0026#34; } } }, { \u0026#34;match\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;value\u0026#34;, \u0026#34;value\u0026#34;: \u0026#34;refs/heads/master\u0026#34;, \u0026#34;parameter\u0026#34;: { \u0026#34;source\u0026#34;: \u0026#34;payload\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;ref\u0026#34; } } } ] } } ] The trigger-rule in config above will ensure that the script is only triggered when header from Github request contains \u0026ldquo;X-Hub-Signature\u0026rdquo; with a secret string and the push has occured in master branch.\nMake sure that the secret string (\u0026ldquo;secret\u0026rdquo; : \u0026ldquo;MyTotallySecretString\u0026rdquo;) is randomly generated. This secret will need to be entered in Github settings as well.\nFor Gitlab and Bitbucket, example hook config can be found on repo page here\nExpose your webhook server safely to the internet There are 2 ways of exposing the webhook server to github:\nProxy using Nginx Via a tunnel e.g by downloading ngrok and then running ngrok http 9000 Nginx configuration Preferably use HTTPS for your domain with Nginx. A good tutorial here.\nExample Nginx config (HTTPS):\nupstream webhook { server localhost:9000; } server { listen 443 ssl http2; server_name YOUR.DOMAIN.COM; ssl_certificate YOUR_CERT_CHAIN; # e.g /etc/letsencrypt/live/DOMAIN/fullchain.pem; ssl_certificate_key YOUR_CERT_KEY; # e.g /etc/letsencrypt/live/DOMAIN/privkey.pem; include /etc/nginx/options-ssl-nginx.conf; ssl_dhparam /etc/nginx/ssl-dhparams.pem; location ~ ^/hooks/(.+)$ { proxy_pass http://webhook; } } Add your webhook URL to Github Go to the settings page of your Github repo then click on Webhook. Enter the URL of your webhook server. If using Nginx, it should be something like https://YOUR.DOMAIN.COM/hooks/deploy-from-git. Make sure you select content type as application/json and secret to the secret you generated earlier.\nBonus: Create a systemd user service for webhook (Linux) Create a systemd unit file with path /home/user/.config/systemd/user/webhook.service . This service does not require sudo/root permissions and can be run by the unprivileged user.\n[Unit] AssertPathExists=/home/user/scripts [Service] WorkingDirectory=/home/user/scripts ExecStart=/home/user/scripts/webhook -hooks hooks.json -hotreload -logfile webhooks.log Restart=always PrivateTmp=true NoNewPrivileges=true [Install] WantedBy=default.target Do systemctl --user daemon-reload and systemctl --user start webhook.service. You can systemctl --user enable webhook.service to ensure that the service always runs when your machine is booted.\n","permalink":"https://rrampage.github.io/2020/05/25/webhook-to-auto-deploy-on-git-push-to-github/","summary":"\u003ch2 id=\"what-is-a-webhook\"\u003eWhat is a webhook?\u003c/h2\u003e\n\u003cp\u003eA \u003ca href=\"https://en.wikipedia.org/wiki/Webhook\"\u003ewebhook\u003c/a\u003e is an endpoint on your server which allows you to execute a particular task. Webhooks are usually triggered by some event. A good use-case for a webhook is running tests on a dedicated test server or deploying your latest \u003ccode\u003emaster\u003c/code\u003e branch to staging/production.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://developer.github.com/webhooks/\"\u003eGithub\u003c/a\u003e / \u003ca href=\"https://docs.gitlab.com/ee/user/project/integrations/webhooks.html\"\u003eGitlab\u003c/a\u003e / Bitbucket allow you to specify a webhook URL in your repository settings. Github triggers the webhook which sends the event data on every push.\u003c/p\u003e\n\u003ch2 id=\"webhook-server\"\u003e\u003ca href=\"https://github.com/adnanh/webhook\"\u003eWebhook\u003c/a\u003e server\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://github.com/adnanh/webhook\"\u003eWebhook\u003c/a\u003e is a very useful golang project which runs any script you specify when a particular endpoint is hit.\u003c/p\u003e\n\u003cp\u003eDownload and extract the binary for your operating system from the \u003ca href=\"https://github.com/adnanh/webhook/releases\"\u003ereleases page\u003c/a\u003e. For Linux, it is \u003ca href=\"https://github.com/adnanh/webhook/releases/download/2.7.0/webhook-linux-amd64.tar.gz\"\u003ehere\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eThe program takes as config a \u003cstrong\u003e\u003ccode\u003ehooks.json\u003c/code\u003e\u003c/strong\u003e file:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-json\" data-lang=\"json\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e[\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"p\"\u003e{\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"nt\"\u003e\u0026#34;id\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;hello-world\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"nt\"\u003e\u0026#34;execute-command\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;/home/user/scripts/hello.sh\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"nt\"\u003e\u0026#34;command-working-directory\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;/home/user/webhook\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"p\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eReplace \u003ccode\u003euser\u003c/code\u003e with the username of your linux user.\u003c/p\u003e\n\u003cp\u003eThe \u003ccode\u003ehello.sh\u003c/code\u003e script.\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"cp\"\u003e#!/bin/bash\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"cp\"\u003e\u003c/span\u003e\u003cspan class=\"nb\"\u003eecho\u003c/span\u003e \u003cspan class=\"s1\"\u003e\u0026#39;Hello!\u0026#39;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eMake the script executable by running \u003cstrong\u003e\u003ccode\u003echmod +x hello.sh\u003c/code\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eStart webhook server as \u003cstrong\u003e\u003ccode\u003ewebhook -hooks hooks.json -hotreload -logfile webhooks.log\u003c/code\u003e\u003c/strong\u003e. The server will run on port 9000 by default. You can check if everything is working by running \u003cstrong\u003e\u003ccode\u003ecurl http://localhost:9000/hooks/hello-world\u003c/code\u003e\u003c/strong\u003e. This will print \u0026ldquo;Hello!\u0026rdquo; in the log file.\u003c/p\u003e","title":"Webhook to auto-deploy on git push to Github"},{"content":"til snippet Here\u0026rsquo;s a snippet I use in my .bashrc file to quickly write down and review TILs (Today I learned):\nTIL_FILE=\u0026#34;$HOME/my-notes-repo/til.md\u0026#34; til () { # checks if the function has been called without any argument if [[ -z $1 ]]; then # opens the file in my editor, setting the cursor to the last line # useful to review latest entries or to write a longer entry vim + \u0026#34;$TIL_FILE\u0026#34;; else # adds a line with today\u0026#39;s date, a TAB and all arguments supplied (\u0026#34;$@\u0026#34;) echo -e \u0026#34;- $( date \u0026#39;+%F\u0026#39; ):\\t$@\u0026#34; \u0026gt;\u0026gt; \u0026#34;$TIL_FILE\u0026#34;; fi } How to use: til to open the file. I use this to review what I learned today or if I need to write a longer, multiline entry til CONTENT to append a line to the file e.g til grep --line-buffered to immediately print especially when tailing files will add this: - 2020-05-23: grep --line-buffered to immediately print especially when tailing files Explanation on the function In case you missed the comments in the function, here\u0026rsquo;s an explanation of what the various lines do:\nif [[ -z $1 ]]; checks if the function has been called without any arguments vim + \u0026quot;$TIL_FILE\u0026quot; opens the file in vim (my preferred editor), setting the cursor to the last line of the file. echo -e \u0026quot;- $tdate:\\t$@\u0026quot; \u0026gt;\u0026gt; \u0026quot;$TIL_FILE\u0026quot;; adds a line with today\u0026rsquo;s date, a TAB and all arguments supplied (\u0026quot;$@\u0026quot;) to til ","permalink":"https://rrampage.github.io/2020/05/23/bash-function-to-add-tils/","summary":"\u003ch2 id=\"til-snippet\"\u003e\u003ccode\u003etil\u003c/code\u003e snippet\u003c/h2\u003e\n\u003cp\u003eHere\u0026rsquo;s a snippet I use in my \u003cstrong\u003e\u003ccode\u003e.bashrc\u003c/code\u003e\u003c/strong\u003e file to quickly write down and review TILs (Today I learned):\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003eTIL_FILE\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$HOME\u003c/span\u003e\u003cspan class=\"s2\"\u003e/my-notes-repo/til.md\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003etil \u003cspan class=\"o\"\u003e()\u003c/span\u003e \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"o\"\u003e{\u003c/span\u003e \n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"c1\"\u003e# checks if the function has been called without any argument\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"k\"\u003eif\u003c/span\u003e \u003cspan class=\"o\"\u003e[[\u003c/span\u003e -z \u003cspan class=\"nv\"\u003e$1\u003c/span\u003e \u003cspan class=\"o\"\u003e]]\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003ethen\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"c1\"\u003e# opens the file in my editor, setting the cursor to the last line\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"c1\"\u003e# useful to review latest entries or to write a longer entry\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        vim + \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$TIL_FILE\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"k\"\u003eelse\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"c1\"\u003e# adds a line with today\u0026#39;s date, a TAB and all arguments supplied (\u0026#34;$@\u0026#34;)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"nb\"\u003eecho\u003c/span\u003e -e \u003cspan class=\"s2\"\u003e\u0026#34;- \u003c/span\u003e\u003cspan class=\"k\"\u003e$(\u003c/span\u003e date \u003cspan class=\"s1\"\u003e\u0026#39;+%F\u0026#39;\u003c/span\u003e \u003cspan class=\"k\"\u003e)\u003c/span\u003e\u003cspan class=\"s2\"\u003e:\\t\u003c/span\u003e\u003cspan class=\"nv\"\u003e$@\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e \u0026gt;\u0026gt; \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$TIL_FILE\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"k\"\u003efi\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"o\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch2 id=\"how-to-use\"\u003eHow to use:\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003e\u003ccode\u003etil\u003c/code\u003e\u003c/strong\u003e to open the file. I use this to review what I learned today  or if I need to write a longer, multiline entry\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003e\u003ccode\u003etil CONTENT\u003c/code\u003e\u003c/strong\u003e to append a line to the file\n\u003cul\u003e\n\u003cli\u003ee.g \u003ccode\u003etil grep --line-buffered to immediately print especially when tailing files\u003c/code\u003e will add this: \u003ccode\u003e- 2020-05-23:   grep --line-buffered to immediately print especially when tailing files\u003c/code\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"explanation-on-the-function\"\u003eExplanation on the function\u003c/h3\u003e\n\u003cp\u003eIn case you missed the comments in the function, here\u0026rsquo;s an explanation of what the various lines do:\u003c/p\u003e","title":"Bash function to add TILs"},{"content":"Setting display brightness using xrandr I auto-adjust the brightness on my display using a cron which runs at 7 pm everyday.\nxrandr --output eDP-1 --brightness 0.9 This sets the brightness to 90% of maximum brightness.\nWorking with Linux Display environments in a cron The above command works when run from my terminal. But when run on a cron, it does not produce any effect. We need to explicitly mention the \u0026ldquo;display number\u0026rdquo; as an environment variable DISPLAY.\n8 19 * * * DISPLAY=:0 xrandr --verbose --output eDP-1 --brightness 0.9 How to get the display number of your monitor? Running xrandr will give you a lot of output like\nScreen 0: minimum 320 x 200, current 1920 x 1080, maximum 16384 x 16384 eDP-1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis) In my case, the display number is :0\nAdjusting backlight brightness vs perceived brightness If xrandr is unable to find backlight, it only changes \u0026ldquo;perceived\u0026rdquo; brightness i.e it does software color correction. If we want to save battery, we will need to reduce the backlight brightness.\nIn Linux, the backlights can be found in /sys/class/backlight. In my case, it was /sys/class/backlight/intel_backlight. In this folder, there are many files like\nactual_brightness max_brightness brightness max_brightness shows highest possible level of brightness for the display. We can adjust the value in brightness file to reduce backlight brightness. Here\u0026rsquo;s what I do to adjust it: echo 1800 \u0026gt; /sys/class/backlight/intel_backlight/brightness (in root crontab)\nUnlike the xrandr command, this does not require setting any DISPLAY variable in the cron file.\nOther tools brightnessctl is an easy to use tool which integrates well with systemd. It is also available in the package managers of various distributions like Debian, Ubuntu, Arch Linux. ","permalink":"https://rrampage.github.io/2020/05/19/setting-display-brightness-on-linux-from-the-command-line/","summary":"\u003ch2 id=\"setting-display-brightness-using-xrandr\"\u003eSetting display brightness using \u003ccode\u003exrandr\u003c/code\u003e\u003c/h2\u003e\n\u003cp\u003eI auto-adjust the brightness on my display using a \u003ca href=\"https://en.wikipedia.org/wiki/Cron\"\u003ecron\u003c/a\u003e which runs at 7 pm everyday.\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003exrandr --output eDP-1 --brightness 0.9\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eThis sets the brightness to 90% of maximum brightness.\u003c/p\u003e\n\u003ch2 id=\"working-with-linux-display-environments-in-a-cron\"\u003eWorking with Linux Display environments in a cron\u003c/h2\u003e\n\u003cp\u003eThe above command works when run from my terminal. But when run on a cron, it does not produce any effect. We need to explicitly mention the \u0026ldquo;display number\u0026rdquo; as an environment variable \u003ccode\u003eDISPLAY\u003c/code\u003e.\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"m\"\u003e8\u003c/span\u003e \u003cspan class=\"m\"\u003e19\u003c/span\u003e * * * \u003cspan class=\"nv\"\u003eDISPLAY\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e:0 xrandr --verbose --output eDP-1 --brightness 0.9\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch3 id=\"how-to-get-the-display-number-of-your-monitor\"\u003eHow to get the display number of your monitor?\u003c/h3\u003e\n\u003cp\u003eRunning \u003ccode\u003exrandr\u003c/code\u003e will give you a lot of output like\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eScreen 0: minimum 320 x 200, current 1920 x 1080, maximum 16384 x 16384\neDP-1 connected primary 1920x1080+0+0 (normal left inverted right x axis y axis)\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eIn my case, the display number is \u003ccode\u003e:0\u003c/code\u003e\u003c/p\u003e\n\u003ch3 id=\"adjusting-backlight-brightness-vs-perceived-brightness\"\u003eAdjusting backlight brightness vs perceived brightness\u003c/h3\u003e\n\u003cp\u003eIf \u003ccode\u003exrandr\u003c/code\u003e is unable to find backlight, it only changes \u0026ldquo;perceived\u0026rdquo; brightness i.e it does \u003ca href=\"https://wiki.archlinux.org/index.php/backlight#Color_correction\"\u003esoftware color correction\u003c/a\u003e. If we want to save battery, we will need to reduce the backlight brightness.\u003c/p\u003e","title":"Setting display brightness on Linux from the command line"},{"content":"Why Wireguard and How does it work? Simple (one config file on server and client and one command to run) Small codebase (5000 LOC) unlike OpenVPN and IPSEC Scheduled to be merged into Linux kernel soon Wireguard works by creating a tunnel to the server using a separate network interface on your machine. You do not need to worry about Proxy settings and all your applications will work without issue.\nInstallation and Key Generation Do this on both server and client machines\n#Installation: For both machines sudo apt install wireguard wireguard-tools openresolv # openresolv is in case you are using systemd-resolved which is default for DNS resolution on Ubuntu # Generate private and public key pair : For both machines umask 077 wg genkey | tee privatekey | wg pubkey \u0026gt; publickey Configure Server Create file called /etc/wireguard/wg0.conf\n[Interface] PrivateKey = \u0026lt;Private Key\u0026gt; ListenPort = \u0026lt;UDP Port to listen on, by default it is 51820\u0026gt; Address = 192.168.2.1/24, fd86:ea04:1115::1/64 ListenPort = 51820 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE SaveConfig = true [Peer] PublicKey = \u0026lt;Client Public Key\u0026gt; AllowedIPs = 192.168.2.2/32 Note that you have to paste the contents of the private key in the conf file, not the path. The PostUp sets up forwarding rules when the wiregaurd interface is started. PostDown deletes the rules when the interface is shutdown. Packet forwarding is essential if you want to use your server as a VPN to the general internet. Enable it on your server by adding the following to /etc/sysctl.conf and run sudo sysctl -p to reload the configuration changes.\n# Add to end of /etc/sysctl.conf net.ipv4.ip_forward=1 net.ipv6.conf.all.forwarding=1 Whitelist the UDP port so that it is reachable by client\nsudo ufw allow 51820/udp sudo ufw enable Configure Client Create file called /etc/wireguard/wg0.conf\n[Interface] PrivateKey = \u0026lt;Private Key\u0026gt; Address = 192.168.2.2/32 SaveConfig = true [Peer] PublicKey = \u0026lt;Server Public Key\u0026gt; AllowedIPs = 0.0.0.0/0, ::/0 Endpoint = \u0026lt;Server Public IP: Port e.g 23.23.23.23:2323\u0026gt; Note that the Address in interface part of client must be within range of AllowedIPs in server\u0026rsquo;s config. If you are not using the server as a VPN, you can restrict the AllowedIPs section of client.\nStart Server and Client Run sudo wg-quick up wg0 Enable it to run as service on system startup using sudo systemctl enable wg-quick@wg0 If you want to stop, use sudo wg-quick down wg0\nTest connection # Ping Google ping -c3 8.8.8.8 sudo wg # shows tyou status of the wireguard connection Resources Whitepaper Adapted from OSX Android client More platforms ","permalink":"https://rrampage.github.io/2018/12/14/easy-vpn-using-wireguard/","summary":"\u003ch2 id=\"why-wireguard-and-how-does-it-work\"\u003eWhy Wireguard and How does it work?\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eSimple (one config file on server and client and one command to run)\u003c/li\u003e\n\u003cli\u003eSmall codebase (5000 LOC) unlike OpenVPN and IPSEC\u003c/li\u003e\n\u003cli\u003eScheduled to be merged into Linux kernel soon\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eWireguard works by creating a tunnel to the server using a separate network interface on your machine.\nYou do not need to worry about Proxy settings and all your applications will work without issue.\u003c/p\u003e\n\u003ch2 id=\"installation-and-key-generation\"\u003eInstallation and Key Generation\u003c/h2\u003e\n\u003cp\u003eDo this on both server and client machines\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e#Installation: For both machines\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003esudo apt install wireguard wireguard-tools openresolv\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# openresolv is in case you are using systemd-resolved which is default for DNS resolution on Ubuntu\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# Generate private and public key pair : For both machines\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nb\"\u003eumask\u003c/span\u003e \u003cspan class=\"m\"\u003e077\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003ewg genkey \u003cspan class=\"p\"\u003e|\u003c/span\u003e tee privatekey \u003cspan class=\"p\"\u003e|\u003c/span\u003e wg pubkey \u0026gt; publickey\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch2 id=\"configure-server\"\u003eConfigure Server\u003c/h2\u003e\n\u003cp\u003eCreate file called \u003ccode\u003e/etc/wireguard/wg0.conf\u003c/code\u003e\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e[Interface]\nPrivateKey = \u0026lt;Private Key\u0026gt;\nListenPort = \u0026lt;UDP Port to listen on, by default it is 51820\u0026gt;\nAddress = 192.168.2.1/24, fd86:ea04:1115::1/64\nListenPort = 51820\nPostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE; ip6tables -A FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE\nPostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE; ip6tables -D FORWARD -i wg0 -j ACCEPT; ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE\nSaveConfig = true\n\n[Peer]\nPublicKey = \u0026lt;Client Public Key\u0026gt;\nAllowedIPs = 192.168.2.2/32\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eNote that you have to paste the \u003cstrong\u003econtents\u003c/strong\u003e of the private key in the conf file, not the path.\nThe \u003cstrong\u003ePostUp\u003c/strong\u003e sets up forwarding rules when the wiregaurd interface is started. \u003cstrong\u003ePostDown\u003c/strong\u003e deletes the rules when the interface is shutdown.\nPacket forwarding is essential if you want to use your server as a VPN to the general internet. Enable it on your server by adding the following to \u003ccode\u003e/etc/sysctl.conf\u003c/code\u003e and run \u003ccode\u003esudo sysctl -p\u003c/code\u003e to reload the configuration changes.\u003c/p\u003e","title":"Easy VPN using Wireguard"},{"content":"I read a post on dev.to which shows how to create git tags using GUI-based git clients. I think that tags are useful to know even when using the git cli.\nWhat are tags Tags are specific points in your code history which are useful to re-visit later e.g you just released a new version of your app. You can tag the commit as v1.0 using git tag v1.0. Anytime you want to reproduce bugs encountered on that version, simply do git checkout v1.0 and investigate.\nHow to use git tag better Checkout code to the tag The tag is linked to the specific commit and not to a branch. When you checkout the tag, git tells you that you are in \u0026ldquo;detached HEAD\u0026rdquo; state. Do not worry, all it means is that you need to create a new branch if you want to retain any changes you make after checking out the tag.\nCreate a new branch exactly at the commit of the tag using git checkout -b BRANCH_NAME TAG_NAME\nMake your tag more informational! You can add more information using git tag -a TAG_NAME -m 'MESSAGE' The tag information can be viewed without having to checkout the tag using git show TAG_NAME.\nCreate tag at a particular commit You do not always have to be at the HEAD or in the tip of the branch to create a tag. If you want to create tag say 5 commits before HEAD, you can use git log to get the correct commit hash e.g git log --pretty=oneline -10 which shows the last 10 commits on the current branch.\nThen, create tag using git tag -a TAG_NAME -m 'MESSAGE' COMMIT_HASH\nList and Delete tags Listing all tags is simply: git tag Similarly, delete a tag using git tag -d TAG_NAME\nShare your tag with others! The tag created is not pushed to remote automatically. If you want your tags to be used by other contributors too, you need to push them using git push origin TAG_NAME\nSign your tags! If you are working on a major project and want to show without any doubt that you have worked on the release, you can sign it using your GPG private key as git tag -s TAG_NAME -m 'MESSAGE'. Anyone who runs git show TAG_NAME on the tag will also see your public key signature along with the tag information.\nThey can additionally verify the tag using git tag -v TAG_NAME. This checks using your public key whether the signature is indeed yours.\nTags vs Branches Why use tags when you have branches? Because branches can change and tags are linked to a specific commit. Thus, marking releases with a tag will give you the state of code when the particular software release was done.\nSummary Tags are great way to remember specific commits It is good to add a message to the tag so that people can easily see why you tagged a particular commit References The Git book chapters:\nTagging Signing ","permalink":"https://rrampage.github.io/2018/11/22/git-tagging-tutorial/","summary":"\u003cp\u003eI read a post on \u003ca href=\"https://dev.to/zellwk/git-tags--37a\"\u003edev.to\u003c/a\u003e which shows how to create git tags using GUI-based git clients. I think that tags are useful to know even when using the git cli.\u003c/p\u003e\n\u003ch2 id=\"what-are-tags\"\u003eWhat are tags\u003c/h2\u003e\n\u003cp\u003eTags are specific points in your code history which are useful to re-visit later e.g\nyou just released a new version of your app. You can tag the commit as \u003ccode\u003ev1.0\u003c/code\u003e using \u003cstrong\u003e\u003ccode\u003egit tag v1.0\u003c/code\u003e\u003c/strong\u003e. Anytime you want to reproduce bugs encountered on that version,  simply do \u003cstrong\u003e\u003ccode\u003egit checkout v1.0\u003c/code\u003e\u003c/strong\u003e and investigate.\u003c/p\u003e\n\u003ch2 id=\"how-to-use-git-tag-better\"\u003eHow to use \u003ccode\u003egit tag\u003c/code\u003e better\u003c/h2\u003e\n\u003ch3 id=\"checkout-code-to-the-tag\"\u003eCheckout code to the tag\u003c/h3\u003e\n\u003cp\u003eThe tag is linked to the specific commit and not to a branch. When you checkout the tag, git tells you that you are in \u0026ldquo;detached HEAD\u0026rdquo; state. Do not worry, all it means is that you need to create a new branch if you want to retain any changes you make after checking out the tag.\u003c/p\u003e\n\u003cp\u003eCreate a new branch exactly at the commit of the tag using \u003cstrong\u003e\u003ccode\u003egit checkout -b BRANCH_NAME TAG_NAME\u003c/code\u003e\u003c/strong\u003e\u003c/p\u003e","title":"Git Tagging Tutorial"},{"content":"Today, I was curious to find out how many programs ending with cat I had on my system. I remember using cat, zcat and wanted to know if there are similar programs. Pressing TAB after cat only gives programs which start with cat.\n1. apropos In my last post, I had mentioned that apropos is a way to search for what programs are available on your system. The search string for apropos can be any regex. So, apropos 'cat$' should solve the problem. cat$ means all words ending with \u0026lsquo;cat\u0026rsquo;\nThe output has\nSTAILQ_CONCAT (3) - implementations of singly-linked lists, singly-linked tail queues, lists and tail queues OPENSSL_strlcat (3ssl) - Memory allocation functions .. bzcat (1) - decompresses files to stdout cat (1) - concatenate files and print on the standard output fc-cat (1) - read font information cache files gencat (1) - Generate message catalog Clearly, the top 2 do not look like programs. Why is apropos then returning them? Let\u0026rsquo;s have a look at the apropos manual\nman apropos # apropos - search the manual page names and descriptions So apropos searches the man pages. And looks like there are man pages for other things and not just programs\u0026hellip;\nDigging deeper, let\u0026rsquo;s try manual for the man pages!\nman man # The table below shows the section numbers of the manual followed by the types of pages they contain. # 1 Executable programs or shell commands # 2 System calls (functions provided by the kernel) # 3 Library calls (functions within program libraries) # 4 Special files (usually found in /dev) # 5 File formats and conventions eg /etc/passwd # 6 Games # 7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7) # 8 System administration commands (usually only for root) # 9 Kernel routines [Non standard] Ok. We are interested in are executable programs i.e section 1 of the man pages. apropos has a way to limit which sections we search using-s flag. apropos -s 1 'cat$' gives us all programs ending with name cat which have an entry in the man pages but it does not show us any programs which do not have a man page.\n2. List all programs on your path The way Bash knows which programs can be called directly by their name (e.g ls) and not by their full path (e.g /usr/bin/ls) is by looking at the $PATH environment variable.\n** Listing all executable files on path **\nHere\u0026rsquo;s a small bash snippet which lists the executable files in PATH (let\u0026rsquo;s call it paths.sh)\n#!/bin/bash # The directories in $PATH are separated by \u0026#34;:\u0026#34;, so we split by it to get individual directories for pdir in $(echo \u0026#34;$PATH\u0026#34; | tr \u0026#34;:\u0026#34; \u0026#34;\\n\u0026#34;) do # We `find` all files in the directory which are executable and print the filename find \u0026#34;$pdir\u0026#34; -maxdepth 1 -executable -type f -printf \u0026#34;%f\\n\u0026#34; done If you prefer Python, here\u0026rsquo;s a small Python program for the same (let\u0026rsquo;s call it paths.py)\nfrom itertools import chain import os path_dirs = os.environ[\u0026#39;PATH\u0026#39;].split(\u0026#39;:\u0026#39;) # Split PATH by \u0026#39;:\u0026#39; print(path_dirs) all_files = chain(*(os.walk(p) for p in path_dirs)) # Iterable of all files in the directories contained in PATH is_exec = lambda x : os.access(x, os.X_OK) # Function to check if a filename is executable execs = chain(*([f for f in fs if is_exec(os.path.join(r,f))] for r,_,fs in all_files)) for x in execs: print(x) Running either our Bash or Python scripts will give us the correct output!\nsh paths.sh | grep \u0026#39;cat$\u0026#39; python3 paths.py | grep \u0026#39;cat$\u0026#39; 3. Power of Bash Completion! When I press TAB TAB after typing a letter, I get a list of suggestions. How does Bash do that? The Bash manual says that it uses complete and compgen built-ins for suggesting completions.\ncompgen generates completions using a list of words (-W) or list of commands (-c). The latter is of particular interest to us. compgen -c prints every executable on our path and all shell built-ins and shell-functions.\ncompgen --help prints following message: compgen: compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlist] [-F function] [-C command] [-X filterpat] [-P prefix] [-S suffix] [word] Display possible completions depending on the options.\nThe options stand for:\na : aliases b : shell builtins c : executable commands d : directories in current directory e : export variables f : files in current directory g : groups in system j : pending jobs (in background / stopped) k : Bash keywords s : System services u : users v : All shell variables So compgen -c | grep 'cat$' should give us every single executable ending with cat.\nEpilogue Diving into this rabbit-hole has given me a better understanding of how Bash completion works, how apropos finds relevant programs and why man pages are organized into various sections.\n","permalink":"https://rrampage.github.io/2018/11/15/bash-command-completion-finding-all-the-cats-in-your-path/","summary":"\u003cp\u003eToday, I was curious to find out \u003cstrong\u003ehow many programs ending with \u003ccode\u003ecat\u003c/code\u003e\u003c/strong\u003e I had on my system. I remember using \u003ccode\u003ecat\u003c/code\u003e, \u003ccode\u003ezcat\u003c/code\u003e and wanted to know if there are similar programs. Pressing TAB after \u003ccode\u003ecat\u003c/code\u003e only gives programs which start with \u003ccode\u003ecat\u003c/code\u003e.\u003c/p\u003e\n\u003ch2 id=\"1-apropos\"\u003e1. \u003ccode\u003eapropos\u003c/code\u003e\u003c/h2\u003e\n\u003cp\u003eIn my \u003ca href=\"https://dev.to/rrampage/getting-help-in-the-terminal-4cmo\"\u003elast post\u003c/a\u003e, I had mentioned that apropos is a way to search for what programs are available on your system. The search string for \u003ccode\u003eapropos\u003c/code\u003e can be any regex. So, \u003ccode\u003eapropos 'cat$'\u003c/code\u003e should solve the problem. \u003ccode\u003ecat$\u003c/code\u003e means all words ending with \u0026lsquo;cat\u0026rsquo;\u003c/p\u003e\n\u003cp\u003eThe output has\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eSTAILQ_CONCAT (3)     - implementations of singly-linked lists, singly-linked tail queues, lists and tail queues\nOPENSSL_strlcat (3ssl) - Memory allocation functions\n..\nbzcat (1)            - decompresses files to stdout\ncat (1)              - concatenate files and print on the standard output\nfc-cat (1)           - read font information cache files\ngencat (1)           - Generate message catalog\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eClearly, the top 2 do not look like programs. Why is \u003ccode\u003eapropos\u003c/code\u003e then returning them?\nLet\u0026rsquo;s have a look at the \u003ccode\u003eapropos\u003c/code\u003e manual\u003c/p\u003e","title":"Bash Command Completion - Finding all the cats in your $PATH!"},{"content":"The command-line is often a daunting place for beginners. With nothing but a blinking cursor and an unfriendly dark screen staring back at you, despair sets in.\nHere are a few things to do if you are stuck:\n1. How to find programs for specific tasks Many times, you want to do a particular task but you do not remember the name of the program which does that. Many shell utilities are not easy to remember at first with their cryptic 2 letter names.\nWorry not, there is a command called apropos to help you out!\nFor example, you want to show a file with line numbers. You know that there is a program but have forgotten its name. You can just type apropos -a line number to get a list of programs. We use -a flag so that apropos will only return those programs which have the words line AND number in their description.\nOn my computer, it gives the following output:\napropos -a line number # Output: addr2line (1) - convert addresses into file names and line numbers. nl (1) - number lines of files x86_64-linux-gnu-addr2line (1) - convert addresses into file names and line numbers. There are 3 programs which have something to do with numbering lines. Let\u0026rsquo;s find out what each of these programs does.\n2. Find out what a program does Let\u0026rsquo;s have a look at our first candidate addr2line. Typing --help after the command is one way of finding out what a program does. It is a convention for command-line programs to print a small help message when you call them with --help. Some programs also allow do the same on calling with -h.\naddr2line --help # Output: Usage: addr2line [option(s)] [addr(s)] Convert addresses into line number/file name pairs. If no addresses are specified on the command line, they will be read from stdin .... Ok. This does program does something with addresses, not what we are looking for. Moving on to the next in our list: nl\nnl --help # Output: Usage: nl [OPTION]... [FILE]... Write each FILE to standard output, with line numbers added. ... Looks like we found our program. Let\u0026rsquo;s try it out. This is our file foo.txt\ncat foo.txt # Output: one two three four five nl foo.txt # Output: 1 one 2 two 3 three 4 four 5 five It works ok but why is it not numbering blank lines?\n3. Getting detailed information about a program The --help messages are very concise. If we want to know in detail what a program does, we need to consult the manual or man pages. These pages are very detailed documentation of what a program does, all its possible options and arguments. Type man nl and have a look at the information\u0026hellip;\nWoah! The terminal screen is filled with information!. You can navigate up and down using the arrow keys and press q to quit.\nWe can see the following in the output:\n. . -b, --body-numbering=STYLE use STYLE for numbering body lines . . STYLE is one of: a number all lines t number only nonempty lines n number no lines So, nl -b a foo.txt will number all lines in foo.txt which is what we want.\nnl -b a foo.txt # Output: 1 one 2 two 3 three 4 5 four 6 five That seems to solve our problem.\nThere is also info command which gives even more detailed usage information. You can try it out using info nl.\n4. Getting examples for using a program But what if we just quickly want to get our command to work without having to read a huge wall of text?\nTurns out, there is a nifty utility you can install called tldr. If you use node or python you can install using npm install -g tldr or pip install tldr respectively. The tldr page has a list of other installation options. After installing it, just try out tldr nl in your terminal\ntldr nl # Output: nl A utility for numbering lines, either from a file, or from standard input. - Number non-blank lines in a file: nl file .... - Number all lines including blank lines: nl -b a file There! We can see the example with easy to understand description.\ntldr is community-driven! People contribute examples for various commands. If your favorite command does not have an entry, you can submit a pull request.\nRecap To recap, we can:\nFind out possible programs which do particular tasks using apropos Get help information for $program using $program --help or man $program Get examples using tldr $program PS: What if we are not able to find programs for our tasks In Section 1, we assume that we will be able to find programs for our task. But many times, we may not have it installed. In such cases, we can search our distribution\u0026rsquo;s package manager e.g apt-get for Ubuntu or yum or dnf for CentOS / Fedora.\nIn Ubuntu, we can search all available packages in the repositories using apt-cache search $KEYWORD.\n","permalink":"https://rrampage.github.io/2018/11/11/command-line-productivity-tips-getting-help-in-the-terminal/","summary":"\u003cp\u003eThe command-line is often a daunting place for beginners. With nothing but a blinking cursor and an unfriendly dark screen staring back at you, despair sets in.\u003c/p\u003e\n\u003cp\u003eHere are a few things to do if you are stuck:\u003c/p\u003e\n\u003ch2 id=\"1-how-to-find-programs-for-specific-tasks\"\u003e1. How to find programs for specific tasks\u003c/h2\u003e\n\u003cp\u003eMany times, you want to do a particular task but you do not remember the name of the program which does that. Many shell utilities are not easy to remember at first with their cryptic 2 letter names.\u003c/p\u003e\n\u003cp\u003eWorry not, there is a command called \u003cstrong\u003e\u003ccode\u003eapropos\u003c/code\u003e\u003c/strong\u003e to help you out!\u003c/p\u003e\n\u003cp\u003eFor example, you want to \u003cstrong\u003eshow a file with line numbers\u003c/strong\u003e. You know that there is a program but have forgotten its name. You can just type \u003ccode\u003eapropos -a line number\u003c/code\u003e to get a list of programs. We use \u003ccode\u003e-a\u003c/code\u003e flag so that \u003ccode\u003eapropos\u003c/code\u003e will only return those programs which have the words \u003cstrong\u003e\u003ccode\u003eline\u003c/code\u003e AND \u003ccode\u003enumber\u003c/code\u003e\u003c/strong\u003e in their description.\u003c/p\u003e\n\u003cp\u003eOn my computer, it gives the following output:\u003c/p\u003e","title":"Command-line productivity tips : Getting help in the terminal"},{"content":"This is useful when you have forked a repository (repo), cloned it to your local machine and want to keep it in sync with the original repo.\nAdding the remote repo We can list the remote repositories for our repo with git remote -v and add the original repo as follows:\ngit remote add upstream LINK_TO_ORIGINAL_REPO Note that it is merely a convention to call it upstream. You can give any name you want.\nCheck that the repo is added to your remote using git remote -v.\nSync with remote Check if there are any changes in remote not on your fork using git fetch upstream Checkout whichever branch you are interested ( git checkout $BRANCH) Merge with upstream using: git merge upstream/$BRANCH Push your changes to origin if needed: git push origin $BRANCH Removing the remote repo If you no longer want to get changes from the remote repo, it is easy to remove it using git remote remove upstream\nReferences Github\u0026rsquo;s help page\n","permalink":"https://rrampage.github.io/2018/11/09/git-how-to-keep-your-fork-updated-with-remote-repository/","summary":"\u003cp\u003eThis is useful when you have forked a repository (repo), cloned it to your local machine and want to keep it in sync with the original repo.\u003c/p\u003e\n\u003ch2 id=\"adding-the-remote-repo\"\u003eAdding the remote repo\u003c/h2\u003e\n\u003cp\u003eWe can list the remote repositories for our repo with \u003cstrong\u003e\u003ccode\u003egit remote -v\u003c/code\u003e\u003c/strong\u003e and add the original repo as follows:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003egit remote add upstream LINK_TO_ORIGINAL_REPO\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eNote that it is merely a convention to call it \u003cstrong\u003e\u003ccode\u003eupstream\u003c/code\u003e\u003c/strong\u003e. You can give any name you want.\u003c/p\u003e\n\u003cp\u003eCheck that the repo is added to your remote using \u003cstrong\u003e\u003ccode\u003egit remote -v\u003c/code\u003e\u003c/strong\u003e.\u003c/p\u003e\n\u003ch2 id=\"sync-with-remote\"\u003eSync with remote\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eCheck if there are any changes in remote not on your fork using \u003cstrong\u003e\u003ccode\u003egit fetch upstream\u003c/code\u003e\u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003eCheckout whichever branch you are interested ( \u003cstrong\u003e\u003ccode\u003egit checkout $BRANCH\u003c/code\u003e\u003c/strong\u003e)\u003c/li\u003e\n\u003cli\u003eMerge with upstream using: \u003cstrong\u003e\u003ccode\u003egit merge upstream/$BRANCH\u003c/code\u003e\u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003ePush your changes to origin if needed: \u003cstrong\u003e\u003ccode\u003egit push origin $BRANCH\u003c/code\u003e\u003c/strong\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"removing-the-remote-repo\"\u003eRemoving the remote repo\u003c/h2\u003e\n\u003cp\u003eIf you no longer want to get changes from the remote repo, it is easy to remove it using \u003cstrong\u003e\u003ccode\u003egit remote remove upstream\u003c/code\u003e\u003c/strong\u003e\u003c/p\u003e","title":"Git: How to keep your fork updated with remote repository"},{"content":"Many times, we have shell scripts which perform some important stuff like inserting into database, mailing reports, etc which we want to run exactly one instance of.\nEnter locks! A simple solution is to create a \u0026ldquo;lock file\u0026rdquo; and check if the file exists when the script starts. If the file is already created, it means another instance of that program is running, so we can fail with message \u0026ldquo;Try again later!\u0026rdquo;. Once the script completes running, it will clean-up and delete the lock file.\nLOCK_FILE=a.lock if [ -f \u0026#34;$LOCK_FILE\u0026#34; ]; then # Lock file already exists, exit the script echo \u0026#34;An instance of this script is already running\u0026#34; exit 1 fi # Create the lock file echo \u0026#34;Locked\u0026#34; \u0026gt; \u0026#34;$LOCK_FILE\u0026#34; # Do the normal stuff # clean-up before exit rm \u0026#34;$LOCK_FILE\u0026#34; This looks promising but there are issues with this approach. What happens if the script does not end correctly i.e it exits because of some failure before it reaches the clean-up part of the code? Or if it gets forcibly terminated with Ctrl+C or kill command? In both these cases, the created lock file will not be deleted. So next time you run the script, you will always get an error and will have to manually delete the file.\nThere is another, more subtle error with the above code. A race condition. If two instances of scripts are started around the same time, it is possible that both of them get past the if [ -f \u0026quot;$LOCK_FILE\u0026quot; ] because the second instance may reach that part of the code before the first instance is able to create the lock file. Thus, we have more than one instance running.\nA better lock! Is there a way to create a lock file which is more robust to race conditions and non-standard termination (Ctrl+C, kill command, etc)? Linux offers flock a utility to manage locks from shell scripts. Using flock, we can rewrite the above snippet as follows:\nLOCK_FILE=a.lock exec 99\u0026gt;\u0026#34;$LOCK_FILE\u0026#34; flock -n 99 || exit 1 # Do stuff and exit! The exec 99\u0026gt;\u0026quot;$LOCK_FILE\u0026quot; creates a file descriptor numbered 99 and assigns it to LOCK_FILE. File descriptors (fd) 0, 1, 2 are for stdin, stdout, stderr respectively. We are creating new fd with a high number to ensure that it does not clash with numbered fds opened later-on by script.\nflock -n 99 || exit 1 does 2 things. Firstly, it acquires an exclusive lock on the file descriptor 99 which refers to our LOCK_FILE. This operation is guaranteed by the linux kernel to be atomic. Secondly, if it fails to acquire the lock, it exits with return code 1. We do not need to worry about any clean up. flock will automatically release the lock when the script exits regardless of how it terminates. This solves our problem!\nWhat if I wanted to add a more informational message instead of exiting directly on failure to acquire lock? We can change the line flock -n 99 || exit 1 as follows:\nflock -n 99 RC=$? if [ \u0026#34;$RC\u0026#34; != 0 ]; then # Send message and exit echo \u0026#34;Already running script. Try again after sometime\u0026#34; exit 1 fi The flock man page has an example which you can use to add an exclusive lock to start of any shell script:\n[ \u0026#34;${FLOCKER}\u0026#34; != \u0026#34;$0\u0026#34; ] \u0026amp;\u0026amp; exec env FLOCKER=\u0026#34;$0\u0026#34; flock -en \u0026#34;$0\u0026#34; \u0026#34;$0\u0026#34; \u0026#34;$@\u0026#34; || : This boilerplate uses the script file itself as a lock. It works by setting an environment variable $FLOCKER to script file name and executing the script with its original parameters after acquiring the lock. On failure however, it does not print anything and silently exits.\n$0 here stands for name of the script. $@ stands for all arguments passed to the script when it was called.\nUse case for me My team uses a test machine where we deploy multiple branches of a code-base. We need to make sure that exactly one person is building the project at a particular time. The deploy script pulls the specified branch of code from git and builds the project, deploys the main service and starts ancillary services. The script takes sometime to execute. If someone tries to deploy another branch while a build is ongoing, both can fail.\nWith the above snippet, calling the script more than once shows the current branch being built and exits with failure.\nFurther reading Flock man page Pitfalls of creating a lock file like in our initial snippet ","permalink":"https://rrampage.github.io/2018/11/06/ensuring-that-a-shell-script-runs-exactly-once/","summary":"\u003cp\u003eMany times, we have shell scripts which perform some important stuff like inserting into database, mailing reports, etc which we want to run exactly one instance of.\u003c/p\u003e\n\u003ch2 id=\"enter-locks\"\u003eEnter locks!\u003c/h2\u003e\n\u003cp\u003eA simple solution is to create a \u0026ldquo;lock file\u0026rdquo; and check if the file exists when the script starts. If the file is already created, it means another instance of that program is running, so we can fail with message \u0026ldquo;Try again later!\u0026rdquo;. Once the script completes running, it will clean-up and delete the lock file.\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-bash\" data-lang=\"bash\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003eLOCK_FILE\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003ea.lock\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003eif\u003c/span\u003e \u003cspan class=\"o\"\u003e[\u003c/span\u003e -f \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$LOCK_FILE\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e \u003cspan class=\"o\"\u003e]\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e \u003cspan class=\"k\"\u003ethen\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"c1\"\u003e# Lock file already exists, exit the script\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"nb\"\u003eecho\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;An instance of this script is already running\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"nb\"\u003eexit\u003c/span\u003e \u003cspan class=\"m\"\u003e1\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003efi\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# Create the lock file\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nb\"\u003eecho\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;Locked\u0026#34;\u003c/span\u003e \u0026gt; \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$LOCK_FILE\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# Do the normal stuff\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# clean-up before exit\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003erm \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$LOCK_FILE\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eThis looks promising but there are issues with this approach. What happens if the script does not end correctly i.e it exits because of some failure before it reaches the clean-up part of the code? Or if it gets forcibly terminated with \u003ccode\u003eCtrl+C\u003c/code\u003e or \u003ccode\u003ekill\u003c/code\u003e command? In both these cases, the created lock file will not be deleted. So next time you run the script, you will always get an error and will have to manually delete the file.\u003c/p\u003e","title":"Ensuring that a shell script runs exactly once"},{"content":"When your Linux machine runs out of memory, Out of Memory (OOM) killer is called by kernel to free some memory. It is often encountered on servers which have a number of memory intensive processes running. In this post, we dig a little deeper into when does OOM killer get called, how it decides which process to kill and if we can prevent it from killing important processes like databases.\nHow does OOM Killer choose which process to kill? The Linux kernel gives a score to each running process called oom_score which shows how likely it is to be terminated in case of low available memory. The score is proportional to the amount of memory used by the process. The score is 10 x percent of memory used by process. So the maximum score is 100% x 10 = 1000. In addition, if a process is running as a privileged user, it gets a slightly lower oom_score as compared to same memory usage by a normal user process. In earlier versions of Linux ( v2.6.32 kernel), there was a more elaborate heuristic which calculated this score.\nThe oom_score of a process can be found in the /proc directory. Let\u0026rsquo;s say that the process id (pid) of your process is 42, cat /proc/42/oom_score will give you the process\u0026rsquo; score.\nCan I ensure some important processes do not get killed by OOM Killer? Yes! The OOM killer checks oom_score_adj to adjust its final calculated score. This file is present in /proc/$pid/oom_score_adj. You can add a large negative score to this file to ensure that your process gets a lower chance of being picked and terminated by OOM killer. The oom_score_adj can vary from -1000 to 1000. If you assign -1000 to it, it can use 100% memory and still avoid getting terminated by OOM killer. On the other hand, if you assign 1000 to it, the Linux kernel will keep killing the process even when it uses minimal memory.\nLet\u0026rsquo;s go back to our process with pid 42. Here is how you can change its oom_score_adj:\necho -200 | sudo tee - /proc/42/oom_score_adj We need to do this as root user or sudo because Linux does not allow normal users to reduce the OOM score. You can increase the OOM score as a normal user without any special permissions. e.g echo 100 \u0026gt; /proc/42/oom_score_adj\nThere is also another, less fine-grained score called oom_adj which varies from -16 to 15. It is similar to oom_score_adj. In fact, when you set oom_score_adj, the kernel automatically scales it down and calculates oom_adj. oom_adj has a magic value of -17 which indicates that the given process should never be killed by OOM killer.\nDisplay OOM scores of all running processes This script displays the OOM score and OOM adjusted score of all running processes, in descending order of OOM score\n#!/bin/bash # Displays running processes in descending order of OOM score printf \u0026#39;PID\\tOOM Score\\tOOM Adj\\tCommand\\n\u0026#39; while read -r pid comm; do [ -f /proc/$pid/oom_score ] \u0026amp;\u0026amp; [ $(cat /proc/$pid/oom_score) != 0 ] \u0026amp;\u0026amp; printf \u0026#39;%d\\t%d\\t\\t%d\\t%s\\n\u0026#39; \u0026#34;$pid\u0026#34; \u0026#34;$(cat /proc/$pid/oom_score)\u0026#34; \u0026#34;$(cat /proc/$pid/oom_score_adj)\u0026#34; \u0026#34;$comm\u0026#34;; done \u0026lt; \u0026lt;(ps -e -o pid= -o comm=) | sort -k 2nr Check if any of your processes have been OOM-killed The easiest way is to grep your system logs. In Ubuntu: grep -i kill /var/log/syslog. If a process has been killed, you may get results like my_process invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0\nCaveats of adjusting OOM scores Remember that OOM is a symptom of a bigger problem - low available memory. The best way to solve it is by either increasing the available memory (e.g better hardware) or moving some programs to other machines or by reducing memory consumption of programs (e.g allocate less memory where possible).\nToo much tweaking of the OOM adjusted score will result in random processes getting killed and not being able to free enough memory.\nReferences proc man page https://askubuntu.com/questions/60672/how-do-i-use-oom-score-adj/ Walkthrough on which part of Linux code is called Classic LWN article (a bit dated) Invoking the OOM killer manually ","permalink":"https://rrampage.github.io/2018/10/04/surviving-the-linux-oom-killer/","summary":"\u003cp\u003eWhen your Linux machine runs out of memory, \u003cstrong\u003eOut of Memory (OOM) killer\u003c/strong\u003e is called by kernel to free some memory. It is often encountered on servers which have a number of memory intensive processes running. In this post, we dig a little deeper into when does OOM killer get called, how it decides which process to kill and if we can prevent it from killing important processes like databases.\u003c/p\u003e\n\u003ch2 id=\"how-does-oom-killer-choose-which-process-to-kill\"\u003eHow does OOM Killer choose which process to kill?\u003c/h2\u003e\n\u003cp\u003eThe Linux kernel gives a score to each running process called \u003ccode\u003eoom_score\u003c/code\u003e which shows how likely it is to be terminated in case of low available memory. The score is proportional to the amount of memory used by the process. The score is \u003ccode\u003e10 x percent of memory used by process\u003c/code\u003e. So the maximum score is 100% x 10 = 1000. In addition, if a process is running as a \u003cstrong\u003eprivileged user\u003c/strong\u003e, it gets a \u003cstrong\u003eslightly lower oom_score\u003c/strong\u003e as compared to same memory usage by a normal user process. In earlier versions of Linux ( v2.6.32 kernel), there was a more elaborate heuristic which calculated this score.\u003c/p\u003e","title":"Surviving the Linux OOM Killer"},{"content":"Gray binary code is a way of expressing binary numbers such that consecutive numbers differ in exactly 1 digit. For example, in our conventional binary system, the numbers are\n000 001 010 011 100 101 110 111 and so on In Gray, they are:\n000 001 011 010 110 111 101 100 and so on In first system, when we go from \u0026lsquo;001\u0026rsquo; to \u0026lsquo;010\u0026rsquo;, there are 2 changes namely the unit\u0026rsquo;s place becomes \u0026lsquo;0\u0026rsquo; from \u0026lsquo;1\u0026rsquo; and the next digit becomes \u0026lsquo;1\u0026rsquo; from \u0026lsquo;0\u0026rsquo;. But in Gray\u0026rsquo;s system, \u0026lsquo;001\u0026rsquo; becomes \u0026lsquo;011\u0026rsquo; where there is only 1 change (that of 2nd digit).\nGray codes are used in error correction in communication.\nGenerating Gray codes of length n Is there a property we can use for easily generating the Gray codes of a given length? Yes! In our previous example, we generated all the Gray codes for n=3. Ignoring the most significant bit (MSB), notice how the 4th and 5th numbers are equal in their first 2 digits, as are the 3rd and 6th, 2nd and 7th and 1st and 8th. The last 4 numbers are reflection of the first 4 if we ignore the last digit. But the last digit is 0 for the 1st 4 numbers and 1 for the last 4\u0026hellip; We have a recursive formulation.\nR(0) = [] R(n+1) = 0R(n) + 1R\u0026rsquo;(n) (R\u0026rsquo;(n) = reverse of R(n)) For n=0, we have an empty list. For n+1, we take R(n), prepend 0 to all elements and to this sequence, we add reverse of R(n) prepended with 1.\nThis can be succinctly expressed in Python as:\ndef gray_code(n): if n \u0026lt;= 0: return [] if n == 1: return [\u0026#39;0\u0026#39;, \u0026#39;1\u0026#39;] res = gray_code(n-1) return [\u0026#39;0\u0026#39;+s for s in res] + [\u0026#39;1\u0026#39;+s for s in res[::-1]] The above function returns in correct order all the 2^n Gray codes of length n. We had to add a case for n==1 because we are treating the numbers as strings so we can prepend \u0026lsquo;0\u0026rsquo; or \u0026lsquo;1\u0026rsquo;. As n=0 is an empty list, we need another case where we first add the strings.\nConverting a binary number to Gray code How do we convert a binary number to Gray code e.g what is Gray code equivalent of 7 (111 in binary)? From our earlier example, it is 100 = 4. So we need a function which takes an integer and returns the equivalent Gray code as integer.\nWe can use our recursive formulation from earlier to arrive at an algorithm. Let n = 2^a + b. Here, a is the MSB of n. G(n) is the Gray code of n. From our earlier formula, G(n) = 2^a + G(2^a-1-b) .. because of the reflection property. Thus, we know the value of G(n) at ath digit. We can keep iterating to get the other digits of G(n).\nOur pseudo-code:\ndef bin_to_gray(n): if n == 0: return 0 if n == 1: return 1 a = MSB(n) # Assume MSB function exists. It finds most significant bit of n b = n - 2**a return 2**a + bin_to_gray(2**a-1-b) from math import log2 as l2 # A simple way to find MSB def MSB(n): return int(l2(n)) An even faster way: It turns out that there is an even faster way of getting the nth Gray code from n. G(n) = n xor n/2\nIn C, Java or Python, this is expressed as:\nreturn n ^ (n \u0026gt;\u0026gt; 1) Addendum : Generating all Gray codes Knuth style! The legendary Donald Knuth uses this algorithm to generate all the tuples in his Art of Computer Programming Vol 4A:\npublic static void gen_gray_bin_taocp(int n) { boolean p = false; // parity bit byte[] a = new byte[n]; // each bit is an element in this array int j = 0; while (true) { System.out.println(Arrays.toString(a)); // Will print the number in reverse order p = !p; if (p) j = 0; else { j = 1; // Find min j so that a[j-1] = 1 while (j \u0026lt; n) { if (a[j-1] == 1) break; j++; if (j == n) // Termination condition return; } a[j] = (byte) (1-a[j]); // We flip the element at j } } ","permalink":"https://rrampage.github.io/2018/08/18/algorithms-gray-binary-code/","summary":"\u003cp\u003e\u003ca href=\"https://en.wikipedia.org/wiki/Gray_code\"\u003eGray binary code\u003c/a\u003e is a way of expressing binary numbers such that consecutive numbers differ in exactly 1 digit.\nFor example, in our conventional binary system, the numbers are\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e000\u003c/li\u003e\n\u003cli\u003e001\u003c/li\u003e\n\u003cli\u003e010\u003c/li\u003e\n\u003cli\u003e011\u003c/li\u003e\n\u003cli\u003e100\u003c/li\u003e\n\u003cli\u003e101\u003c/li\u003e\n\u003cli\u003e110\u003c/li\u003e\n\u003cli\u003e111 and so on\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn Gray, they are:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e000\u003c/li\u003e\n\u003cli\u003e001\u003c/li\u003e\n\u003cli\u003e011\u003c/li\u003e\n\u003cli\u003e010\u003c/li\u003e\n\u003cli\u003e110\u003c/li\u003e\n\u003cli\u003e111\u003c/li\u003e\n\u003cli\u003e101\u003c/li\u003e\n\u003cli\u003e100 and so on\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIn first system, when we go from \u0026lsquo;001\u0026rsquo; to \u0026lsquo;010\u0026rsquo;, there are 2 changes namely the unit\u0026rsquo;s place becomes \u0026lsquo;0\u0026rsquo; from \u0026lsquo;1\u0026rsquo; and the next digit becomes \u0026lsquo;1\u0026rsquo; from \u0026lsquo;0\u0026rsquo;.\nBut in Gray\u0026rsquo;s system, \u0026lsquo;001\u0026rsquo; becomes \u0026lsquo;011\u0026rsquo; where there is only 1 change (that of 2nd digit).\u003c/p\u003e\n\u003cp\u003eGray codes are used in error correction in communication.\u003c/p\u003e\n\u003ch3 id=\"generating-gray-codes-of-length-n\"\u003eGenerating Gray codes of length n\u003c/h3\u003e\n\u003cp\u003eIs there a property we can use for easily generating the Gray codes of a given length? Yes! In our previous example, we generated all the Gray codes for n=3. Ignoring the most significant bit (\u003cstrong\u003eMSB\u003c/strong\u003e), notice how the 4th and 5th numbers are equal in their first 2 digits, as are the 3rd and 6th, 2nd and 7th and 1st and 8th. The last 4 numbers are \u003cstrong\u003ereflection\u003c/strong\u003e of the first 4 if we ignore the last digit. But the last digit is 0 for the 1st 4 numbers and 1 for the last 4\u0026hellip; We have a recursive formulation.\u003c/p\u003e","title":"Algorithms: Gray Binary Code"},{"content":"I have started the 100 days of code challenge. I intend to use this time to check out new languages and frameworks and solve some fun problems.\nI will update this post with my logs.\nAug 13 2018 D0 : Algorithms for calculating number of combinations and generating them in a lexicographical increasing order. Blog\nAug 14 2018 D1 : Working on algorithm for generating all permutations. First I managed to generate all possible r repetitions of n i.e n^r. Next, I read up and wrote code on Heap\u0026rsquo;s algorithm. I am still not sure of the intuition behind the algorithm. Also, it does not generate the permutations in lexicographical increasing order.\nAug 15 2018 D2 : Learned and implemented an algorithm that generates all permutations in a lexicographical order. It is not as efficient as Heap\u0026rsquo;s algorithm.\nAug 16 2018 D3 : Stumbled across the game of Set. Wrote a small python script which generates all solutions of any given game.\nAug 17 2018 D4 : Learning to use Puppeteer.js along with Google Cloud Functions. This dev.to post was very useful in getting me started.\nAug 18 2018 D5 : Read about Gray Codes, an alternative way of ordering binary numbers in TAOCP Vol 4. Wrote a blog post exploring some of their properties and how to generate them.\nAug 19 2018 D6 : Wrote a script which calculates iterations for Conway\u0026rsquo;s Game of Life\nAug 20 2018 D7 : Went through some Dynamic programming problems. Staircase problem is a good example of DP.\nAug 21 2018 D8: More recursion and DP. Towers of Hanoi and Longest unique sub-string problem.\nAug 24 2018 D9: After 2 days break due to travel, got back to C and linked lists. I am working through Stanford\u0026rsquo;s Linked List problems.\nAug 25 2018 D10 Working on a Sudoku solver. Made some more progress on the linked list problems from yesterday.\nAug 26 2018 D11 My sudoku solver can solve easy sudokus from Code Wars\nGist\nAug 27 2018 D12 Reading up on back-tracking and n-queens problem.\nAug 28 2018 D13 Some fun with numpy and pandas and setting up virtualenvs.\nAug 29 2018 D14 Solved Run Length Encoding problem recursively\nAug 30 2018 D15 Data visualization using matplotlib in Python\nAug 31 2018 D16 1D Cellular Automata in Java\nSep 1 2018 D17 Solved some regex golf problems\nSep 2 2018 D18 Played around with TMDB API using Python\nSep 3 2018 D19 Some interactive graph visualizations using Gephi\nSep 4 2018 D20 Implemented Tries in Java\nSep 5 2018 D21 Codewars kata on displaying a Tic-tac-toe board in C\nSep 6 2018 D22 Dynamic programming - Coin change problem!\nSep 7 2018 D23 Data cleaning using OpenRefine\nSep 8 2018 D24 Using SQLite for importing tabular data\nSep 9 2018 D25 SQLite window functions\nSep 13 2018 D26 After 3 day break because of job interviews and travel, a simple Python problem on moving 0s to the end of list on Codewars\nSep 14 2018 D27 Full-text search using SQLite (FTS4 and FTS5 engines)\nSep 15 2018 D28 n-Queens in Python using backtracking in just 20 lines! Code here: gist\nSep 16 2018 D29 Playing around with D3.js for data visualization\nSep 17 2018 D30 Creating SVGs of various shapes in HTML. Awesome guide here\nSep 18 2018 D31 Befunge interpreter on Codewars\nSep 19 2018 D32 Learned more about JS Array.reduce\nSep 20 2018 D33 Learned more about git branching and detaching from HEAD commit\nSep 21 2018 D34 Learned to use sed effectively\nSep 22 2018 D35 os.walk in Python is very nifty for crawling the filesystem!\nSep 23 2018 D36 More d3.js, visualizations using choropleths for maps\nSep 24 2018 D37 Learned how OOM killer works on Linux. Wrote an article here\nHave started working once again. I am putting this on hiatus till I find more time. Meanwhile, I will continue posting blog posts on my learning.\n","permalink":"https://rrampage.github.io/2018/08/13/100-days-of-code/","summary":"\u003cp\u003eI have started the \u003ca href=\"https://www.100daysofcode.com/\"\u003e100 days of code challenge\u003c/a\u003e.\nI intend to use this time to check out new languages and frameworks and solve some fun problems.\u003c/p\u003e\n\u003cp\u003eI will update this post with my logs.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAug 13 2018 D0\u003c/strong\u003e : Algorithms for calculating number of combinations and generating them in a lexicographical increasing order. \u003ca href=\"https://dev.to/rrampage/algorithms-generating-combinations-100daysofcode-4o0a\"\u003eBlog\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAug 14 2018 D1\u003c/strong\u003e : Working on algorithm for generating all permutations. First I managed to generate all possible \u003ccode\u003er\u003c/code\u003e repetitions of \u003ccode\u003en\u003c/code\u003e i.e \u003ccode\u003en^r\u003c/code\u003e. Next, I read up and wrote code on Heap\u0026rsquo;s algorithm. I am still not sure of the intuition behind the algorithm. Also, it does not generate the permutations in lexicographical increasing order.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAug 15 2018 D2\u003c/strong\u003e : Learned and implemented an algorithm that generates all permutations in a lexicographical order. It is not as efficient as Heap\u0026rsquo;s algorithm.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAug 16 2018 D3\u003c/strong\u003e : Stumbled across the game of \u003ca href=\"https://www.nytimes.com/crosswords/game/set\"\u003eSet\u003c/a\u003e. Wrote a small python script which generates all solutions of any given game.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAug 17 2018 D4\u003c/strong\u003e : Learning to use \u003ca href=\"https://pptr.dev/\"\u003ePuppeteer.js\u003c/a\u003e along with \u003ca href=\"https://cloud.google.com/functions\"\u003eGoogle Cloud Functions\u003c/a\u003e. This \u003ca href=\"https://dev.to/_mertsimsek/to-use-google-cloud-function-as-google-chrome-3e7e\"\u003edev.to post\u003c/a\u003e was very useful in getting me started.\u003c/p\u003e","title":"100 Days Of Code"},{"content":"In how many different ways can we select r objects from a collection of n objects? In mathematics, this is called combinations.\nThe formula for the number of combinations is: where, n! denotes the factorial of a number that is the product of all numbers from 1 to n (inclusive).\nPrelude : A function for calculating factorial of a number public static long factorial(int n) { long res = 1L; for (int i = 1; i \u0026lt;= n; i++) { res *= i; } return res; } Calculating Combinations That was simple! Let us now move on to calculating the number of combinations given n and r\npublic static long combinations(int n, int r) { if (r \u0026lt; 0) { return 0; } long res = 1L; if (r \u0026gt; n - r) { r = n - r; } for (int i = 0; i \u0026lt; r; i++) { res *= (n - i); res /= (i + 1); } return res; } What does this algorithm do? Recall that we need to find n!/r!(n-r)! which will be of the form n(n-1)...(n-r+1)/1.2...r. Similar to factorial, we initialize the result as 1 and multiply by n-i and divide by i+1. Will this result in a fractional number? No. This is because first, we multiply by n and divide by 1. Next, we multiply by n-1 and divide by 2. Now, either n or n-1 have to be even (as they are consecutive numbers). Similarly, next when we divide by 3, one of n,n-1 and n-2 must be divisible by 3.\nIn the above code, we also make use of the mathematical property that combinations(n,r) = combinations(n,n-r). This way, we can do less number of operations for calculating the combinations.\nGenerating the combinations Counting the number of combinations was not so hard! Now, let\u0026rsquo;s generate all the combinations.\nGiven n and r, we will print out all the combinations. For the n objects, we will use the numbers from 0 to (n-1). Additionally, we will generate them in a lexicographical order which is math speak for sorted order. Finally, in a combination containing a and b, if a \u0026lt; b, we will print a b instead of b a. Formally stated, if a[k] and a[k+1] are the kth and (k+1)th elements in a generated combination, a[k] \u0026lt; a[k+1] for all k For example, given n = 4, r = 2, we have:\n0 1 0 2 0 3 1 2 1 3 2 3 i.e 6 combinations.\nNotice that we have 0 1 and not 1 0. This is because we are generating each combination in lexicographical order and we take the minimum for each combination.\nGenerating for a specific value of r (r = 2) If we have a specific value of r say 2, the code will involve 2 for loops like:\nfor (int i = 0; i \u0026lt; n-1; i++) { for (int j = i+1; j \u0026lt; n; j++) { println(i + \u0026#34; \u0026#34; + j); } } In the code above, our first loop variable i goes from 0 to n-2 and the next variable j goes from i+1 to n-1. Why so? This is because we have a requirement for taking the lexicographical minimum combination, so i \u0026lt; j from our constraints.\nIf value of r is fixed, we can simply create r for loops. But it is not fixed\u0026hellip;\nGeneralizing! Now, let\u0026rsquo;s move on to the main goal - generate combinations of n numbers taken r at a time. This section will be a little verbose as I have outlined how I arrived at the correct code. If you are interested in just the algorithm, feel free to skip to the bottom of the article\nIf we notice our previous code for r = 2, our first combination is always 0 1 as i = 0, j = 1. Similarly, if r was 3, our first combination would be 0 1 2. There is a pattern!\nBy creating an array a of size r, we can generate the first combination as 0 1 2 .. r-1. We have the first combination ready. What about the rest? Somehow, if we increment elements in this array, we will generate the combinations\u0026hellip;\nAgain, looking at the r = 2 case, notice that the last combination is n-2 n-1. Similarly, for r = 3, it is n-3 n-2 n-1. Thus, for r elements, it will be n-r+1 n-r+2 .. n-1. There is one more insight - there is exactly one combination which starts with n-r+1. If our array\u0026rsquo;s first element reaches n-r+1, we are done!\nWe now have a termination condition for our function: a[0] == n-r+1\nThe code we have so far will look like:\nint n; // Given int r; // Given int[] a = new int[r]; // Initialize array of size r for (int i = 0; i \u0026lt; r; i++) { a[i] = i; // Initialize array with first combination } while (a[0] \u0026lt; n-r+1) { // Our termination condition // DO SOMETHING! } We have a while loop that checks for termination condition. For the loop to terminate, we need to steadily progress from our first combination to the last combination. As we are generating elements in lexicographical order, the last element of the array must be incremented first. Then the second from last element and so on.\nIn our earlier example of n = 4, r = 2, we had\n0 1 0 2 0 3 1 2 1 3 2 3 After 0 3, we get 1 2. This means once the r-1 element (last element) reaches its maximum, we increment r-2 element from 0 to 1 and also reset the value of r-1 element to a[r-2]+1 as it must always be at least 1 greater than the r-2 element (from our constraints). Moving to our pseudo-code, let\u0026rsquo;s add this to the while loop\nint i = r-1; // variable i keeps track of which element in array we are incrementing while (a[0] \u0026lt; n-r+1) { // Our termination condition if (i \u0026gt; 0 \u0026amp;\u0026amp; a[i] == n-r+1) { i = i-1; //If a[i] has reached the max allowable value, decrement i and move to next element in array } printArray(a); // pseudo-code to print out the combination a[i] = a[i]+1; // increment if (i \u0026lt; r-1) { a[i+1] = a[i]+1; // Reset `i+1` element as previous element + 1, according to our constraints i = i+1; // Once you have reset the i+1 element, it is no longer \u0026lt; n-r+i and hence, we can move it back to old value } } We have an index variable i which we use to check which is the element in the array to be incremented. In the first if in above code, we check if the a[i] has reached its maximum value of n-r+i. If yes, we decrement i as a[i] can no longer be incremented. Moving out of if, we then print the combination and increment a[i]. Now, if i is no longer r-1 i.e it is no longer last element of a, we must reset it to r-1 and also set the value of a[r-1] as a[r-2]+1. This works for r=2. Hooray! We have abstracted out the for loop in the earlier section into a while loop with a few conditionals.\nBut does this work for r \u0026gt; 2? No\u0026hellip; We need a minor change to make it work! Change the if statements inside the loop to while loops and we are done! In case of first loop, we need to find the maximum i which is less than n+r-i. For example n=5, r=3 we have:\n0 1 2 0 1 3 0 1 4 0 2 3 0 2 4 0 3 4 1 2 3 1 2 4 1 3 4 2 3 4 As we move from 0 3 4 to 1 2 3, both i = 2 (a[2] = 4) and i = 1 (a[1] = 3) are at their maximum. We need to move to i = 0. Similarly, the second if must be a while loop because once we have incremented the a[i] for minimum i, we need to reset the outer elements of array to maintain our constraints.\nOur final code: int[] a = new int[r]; // initialize first combination for (int i = 0; i \u0026lt; r; i++) { a[i] = i; } int i = r - 1; // Index to keep track of maximum unsaturated element in array // a[0] can only be n-r+1 exactly once - our termination condition! while (a[0] \u0026lt; n - r + 1) { // If outer elements are saturated, keep decrementing i till you find unsaturated element while (i \u0026gt; 0 \u0026amp;\u0026amp; a[i] == n - r + i) { i--; } printArray(a); // pseudo-code to print array as space separated numbers a[i]++; // Reset each outer element to prev element + 1 while (i \u0026lt; r - 1) { a[i + 1] = a[i] + 1; i++; } } Proof of termination Now that we have our algorithm, how can we show that it terminates? In each iteration of our outer while loop, we increment the element of the array with maximum index i which has not reached value n-r+i while maintaining our constraints. Due to the lexicographical ordering, our previous combination is always lesser than our currently generated combination. As there are only a finite number of combinations till we reach our \u0026ldquo;last\u0026rdquo; combination, we can say that our algorithm will terminate.\nMeta: I began my 100 days of code challenge today with this problem. I will create a separate post explaining my motivations and plans.\nRegarding this problem statement of generating combinations, I had some trouble initially moving from r=2 case to the general one. It took me some time to find the correct termination condition. I am happy that the final algorithm is relatively compact. I also want to do a proof of correctness for this algorithm later.\n","permalink":"https://rrampage.github.io/2018/08/12/algorithms-generating-combinations/","summary":"\u003cp\u003eIn how many different ways can we select \u003ccode\u003er\u003c/code\u003e objects from a collection of \u003ccode\u003en\u003c/code\u003e objects? In mathematics, this is called \u003ca href=\"https://en.wikipedia.org/wiki/Combination\"\u003ecombinations\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"https://thepracticaldev.s3.amazonaws.com/i/w8utl55xbryxdvolbkdx.jpg\" alt=\"Combinations of 5 objects, taken 2 at a time\"  /\u003e\n\u003c/p\u003e\n\u003cp\u003eThe formula for the number of combinations is:\n\u003cimg loading=\"lazy\" src=\"https://wikimedia.org/api/rest_v1/media/math/render/svg/813f7124a61dac205542db3f8491b36cb306453a\" alt=\"\"  /\u003e\n\nwhere, \u003ccode\u003en!\u003c/code\u003e denotes the factorial of a number that is the product of all numbers from 1 to n (inclusive).\u003c/p\u003e\n\u003ch3 id=\"prelude--a-function-for-calculating-factorial-of-a-number\"\u003ePrelude : A function for calculating factorial of a number\u003c/h3\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kd\"\u003estatic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kt\"\u003elong\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nf\"\u003efactorial\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"kt\"\u003elong\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eres\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e1L\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003efor\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026lt;=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"o\"\u003e++\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"n\"\u003eres\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e*=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003ereturn\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eres\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch3 id=\"calculating-combinations\"\u003eCalculating Combinations\u003c/h3\u003e\n\u003cp\u003eThat was simple! Let us now move on to calculating the number of combinations given \u003ccode\u003en\u003c/code\u003e and \u003ccode\u003er\u003c/code\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kd\"\u003estatic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kt\"\u003elong\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nf\"\u003ecombinations\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003er\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eif\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003er\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026lt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"k\"\u003ereturn\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"kt\"\u003elong\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eres\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e1L\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003eif\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003er\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003er\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"n\"\u003er\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003er\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003efor\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026lt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003er\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"o\"\u003e++\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"n\"\u003eres\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e*=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"n\"\u003eres\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e/=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ei\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e+\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003ereturn\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eres\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eWhat does this algorithm do? Recall that we need to find \u003ccode\u003en!/r!(n-r)!\u003c/code\u003e which will be of the form \u003ccode\u003en(n-1)...(n-r+1)/1.2...r\u003c/code\u003e. Similar to \u003ccode\u003efactorial\u003c/code\u003e, we initialize the result as \u003ccode\u003e1\u003c/code\u003e and multiply by \u003ccode\u003en-i\u003c/code\u003e and divide by \u003ccode\u003ei+1\u003c/code\u003e. Will this result in a fractional number? No. This is because first, we multiply by \u003ccode\u003en\u003c/code\u003e and divide by \u003ccode\u003e1\u003c/code\u003e. Next, we multiply by \u003ccode\u003en-1\u003c/code\u003e and divide by \u003ccode\u003e2\u003c/code\u003e. Now, either \u003ccode\u003en\u003c/code\u003e or \u003ccode\u003en-1\u003c/code\u003e have to be even (as they are consecutive numbers). Similarly, next when we divide by 3, one of \u003ccode\u003en\u003c/code\u003e,\u003ccode\u003en-1\u003c/code\u003e and \u003ccode\u003en-2\u003c/code\u003e must be divisible by 3.\u003c/p\u003e","title":"Algorithms: Generating Combinations"},{"content":"Awk is a small but capable programming language which is used for processing text. It was developed by Aho, Weinberger, Kerninghan at Bell Labs.\nJulia Evans made an awesome Awk comic: Awk scans input file as a sequence of lines and splits each line into fields. The field separator is usually whitespace but you can customize it to any character.\nAn awk program is a sequence of pattern-action pairs i.e for each line, it checks if it matches the pattern and if yes, it performs the associated action on the line. Awk can be used interactively or to run saved programs.\nHere is what Awk does written in Python-like pseudocode:\ninitialize() # Initializes variables in BEGIN block for line in input_lines: # Awk divides file / input into a list of lines for condition, action in conditions: # A program is a list of condition-action pairs if condition(line): #match line against condition action() #perform action on match Here are some small snippets of Awk:\nExample - Hello World! You can run awk programs inline or through a file:\nawk \u0026#39;BEGIN{ print \u0026#34;Hello, World!\u0026#34;}\u0026#39; Alternatively, you can save this to a file hello.awk:\nBEGIN{ print \u0026#34;Hello, World!\u0026#34;} Then run it as awk -f hello.awk\nExample - Reading a CSV and printing a specific column Let\u0026rsquo;s now do something useful! Download this csv which is 2010 census data by zip code in Los Angeles city.\nRead the first 3 lines from csv: head -3 2010_Census_Populations_by_Zip_Code.csv\nZip Code,Total Population,Median Age,Total Males,Total Females,Total Households,Average Household Size 91371,1,73.5,0,1,1,1 90001,57110,26.6,28468,28642,12971,4.4 We will print just the total column using awk -F, '{print $2}' 2010_Census_Populations_by_Zip_Code.csv\nThe -F, sets the field separator to comma as we need to split by commas for getting fields in a CSV file. $n allows you to use the value in the nth column.\nExample - Computing some statistics Awk allows the use of variables and functions. Let\u0026rsquo;s see how to use them by computing the total population in the entire city.\n# total.awk {s += $2} END {print \u0026#34;Total population:\u0026#34;, s} Variables are by default initialized to 0. Here, we use a variable s to hold the total.\nRunning this script as awk -F, -f total.awk 2010_Census_Populations_by_Zip_Code.csv, we get output: Total population: 10603988\nSpecial variables and built-in functions Awk uses some special variables and functions to make your programs more compact:\nNF : Number of fields in a line NR : Line number $0 : The entire input line length : gives number of characters in a string Now, we will compute the average household size which is total population divided by total households. The columns of interest are $2 and $6. We also want the average population per zip code. Our script:\n# stats.awk { s += $2; h += $6;} END {print \u0026#34;Total population:\u0026#34;, s, \u0026#34;\\nTotal households:\u0026#34;, h, \u0026#34;\\nAverage household size:\u0026#34;, s/h, \u0026#34;\\nAverage population per zip code:\u0026#34;, s/NR} NR gives us the total number of lines. But we do not want the header line. We can use tail command to skip the 1st line as tail -n +2. Running tail -n +2 2010_Census_Populations_by_Zip_Code.csv | awk -F, -f total.awk gives us :\nTotal population: 10603988 Total households: 3497698 Average household size: 3.0317 Average population per zip code: 33241.3 Example - Pattern matching We have done some useful things with awk so far, but we have ignored its biggest strength - pattern matching. We can match based on field values, regexes, line numbers.\nPrint every 2nd line : NR%2 == 0 {print $0}. Here $0 stands for the entire line. Print all zip codes with population \u0026gt; 100,000 : $2 \u0026gt; 100000 {print $1} Print all zip codes with population \u0026gt; 10,000 and average household size \u0026gt; 4 : $2 \u0026gt; 10000 \u0026amp;\u0026amp; $7 \u0026gt; 4 { print $1}. We can combine conditions using \u0026amp;\u0026amp; and || which stand for logical and and or respectively. Further reading There is a lot more to Awk. Here are some references:\nThe best resource for learning Awk is The AWK programming language written by the same trio. This book goes over and beyond a typical programming language tutorial and teaches you how to use your Awk superpowers to build versatile systems like a relational database, a parser, an interpreter, etc.\nThe GNU Awk Manual for Effective Awk Programming is a thorough reference.\n","permalink":"https://rrampage.github.io/2018/05/26/awk-a-useful-little-language/","summary":"\u003cp\u003eAwk is a small but capable programming language which is used for processing text. It was developed by Aho, Weinberger, Kerninghan at Bell Labs.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://twitter.com/b0rk\"\u003eJulia Evans\u003c/a\u003e made an awesome Awk \u003ca href=\"https://wizardzines.com/comics/awk/\"\u003ecomic\u003c/a\u003e:\n\n  \u003cimg src=\"https://wizardzines.com/images/uploads/awk.png\" alt=\"AWK comic\"/\u003e\n\u003c/p\u003e\n\u003cp\u003eAwk scans input file as a sequence of lines and splits each line into fields. The field separator is usually whitespace but you can customize it to any character.\u003c/p\u003e\n\u003cp\u003eAn \u003ccode\u003eawk\u003c/code\u003e program is a sequence of \u003cstrong\u003epattern-action\u003c/strong\u003e pairs i.e for each line, it checks if it matches the pattern and if yes, it performs the associated action on the line. Awk can be used interactively or to run saved programs.\u003c/p\u003e\n\u003cp\u003eHere is what Awk does written in Python-like pseudocode:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-python\" data-lang=\"python\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003einitialize\u003c/span\u003e\u003cspan class=\"p\"\u003e()\u003c/span\u003e \u003cspan class=\"c1\"\u003e# Initializes variables in BEGIN block\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003efor\u003c/span\u003e \u003cspan class=\"n\"\u003eline\u003c/span\u003e \u003cspan class=\"ow\"\u003ein\u003c/span\u003e \u003cspan class=\"n\"\u003einput_lines\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"c1\"\u003e# Awk divides file / input into a list of lines\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e    \u003cspan class=\"k\"\u003efor\u003c/span\u003e \u003cspan class=\"n\"\u003econdition\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"n\"\u003eaction\u003c/span\u003e \u003cspan class=\"ow\"\u003ein\u003c/span\u003e \u003cspan class=\"n\"\u003econditions\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e \u003cspan class=\"c1\"\u003e# A program is a list of condition-action pairs\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e        \u003cspan class=\"k\"\u003eif\u003c/span\u003e \u003cspan class=\"n\"\u003econdition\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003eline\u003c/span\u003e\u003cspan class=\"p\"\u003e):\u003c/span\u003e \u003cspan class=\"c1\"\u003e#match line against condition\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e            \u003cspan class=\"n\"\u003eaction\u003c/span\u003e\u003cspan class=\"p\"\u003e()\u003c/span\u003e \u003cspan class=\"c1\"\u003e#perform action on match \u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eHere are some small snippets of Awk:\u003c/p\u003e\n\u003ch2 id=\"example---hello-world\"\u003eExample - Hello World!\u003c/h2\u003e\n\u003cp\u003eYou can run \u003ccode\u003eawk\u003c/code\u003e programs inline or through a file:\u003c/p\u003e","title":"Awk - A useful little language"},{"content":"This is part 2 of my series on JVM Memory management and debugging. Read part 1 here\nIn this post, we will cover symptoms of memory issues for JVM-based applications, which tools we can use to diagnose them and how we can fix them.\nSymptoms Here are a few symptoms of memory issues:\nPoor application performance Abnormal memory usage OutOfMemory errors (OOME) Poor Application Performance Application not performing to expected level Long response times Dropping client requests Stuck threads Service unavailability Large gaps in timestamps in application logs Causes of memory problems: Misconfigured memory Old generation memory space is sized smaller than live-set of objects. This triggers a major garbage collection (GC), resulting in larger pauses. Code cache is smaller than generated compiled code footprint Young generation is not sized appropriately leading to premature promotion of objects PermGen / Metaspace not sized correctly leading to full GC Memory leaks - Unintentional retention of objects in memory spaces Unintentional references to set of objects in heap Not dereferencing classloader instances appropriateky Not releasing native resources appropriately Excessive use of finalizers Objects with finalizers may delay their own GC Finalizer thread needs to invoke finalize() method of the instances before reclaiming them There can only be 1 Finalizer thread. If it does not keep up with rate at which objects become available for finalization, JVM fails with OOME Pending finalizer objects are essentially accumulated garbage Finalizers deprecated in Java 9 Explicit GC calls System.gc() and diagnostic data collections can cause long pauses -XX:+DisableExplicitGC can disable System.gc() calls -XX:+PrintClassHistogram also calls an explicit GC when receiving kill -3 signal OutOfMemoryError Hierarchy : Throwable -\u0026gt; Error -\u0026gt; VirtualMachineError -\u0026gt; OutOfMemoryError (Unchecked exception) Thrown when JVM runs out of space in various memory spaces or cannot proceed further with process execution. Some of the possibilities: Heap space full JVM already invoked full GC but could not free up space Heap may be sized smaller than app footprint or app is unnecessarily holding on to some set of objects in heap GC overhead limit exceeded Too many GCs with very less space claimed Application threads not getting any CPU cycles Requested array size exceeds VM limit PermGen space / Metaspace / compressed class space Full GC invoked but unable to free space in Metaspace and application is attempting to load more classes Metaspace by default \u0026ldquo;unlimited\u0026rdquo; but can be controlled by MaxMetaspaceSize. By default, 1 GB reserved for compressed class space Make sure that -Xnoclassgc is not in use as it prevents unloading of classes Native memory - out of swap space / stack trace with native method Native space used for Java thread stacks, loaded jars, zips, native libraries, native resources like files; mem allocated from native code Unable to allocate more native memory or to create new threads or native memory leaks Running 32 bit JVM on 64 bit machine puts 4 GB limit on process size Position of Java heap can put a cap on max size of native heap. Can be controlled by option -XX:HeapBaseMinAddress=n to specify address native heap should be based at CodeCache warnings warning message printed by JVM saying CodeCache full, compiler has been disabled. No OOME when code cache is full Emergency cleanup undertaken by Sweeper. This may discard compiled code and JIT may need to perform optimizations again Ensure appropriate size of CC using ReservedCodeCacheSize option Direct Buffer Memory ByteBuffer.allocateDirect(N) : Direct buffers which are garbage collected using phantom references and a reference queue Unlimited memory by default but can be controlled by -XX:MaxDirectMemorySize=n Used by Java NIO. Heap ByteBuffer for I/O uses temporary direct ByteBuffer Diagnostic Data, Data Collection and Analysis Tools Troubleshooting Memory leaks Confirm memory leak\nMonitor heap usage over time If full GCs unable to claim space in OldGen, could be config issue Heap size may be too small -\u0026gt; Increase heap size and monitor! If issue persists, it could be a memory leak -XX:+GCTimeLimit sets upper limit on amount of time GCs can spend in percent of total time, default 98% -XX:+GCHeapFreeLimit sets lower limit on amount of space that should be freed by a GC, represented as % of max heap, default is 2% OutOfMemoryError is thrown after a full GC if previous 5 consecutive GCs were not able to keep the GC cost below GCTimeLimit or free up at least GCHeapFreeLimit space PermGen/Metaspace may be too small if frequent Full GCs do not claim any space Diagnostic data and analysis\nGC logs are helpful for determining heap requirements, finding out excessive GCs and long GC pauses and in configuration of memory spaces For Java 9+, G1 options are: -Xlog:gc*,gc+phases=debug:file=gc.log . For non G1, -Xlog:gc*:file=gc.log. For older JVMs, -XX:+PrintGCDetails, -XX:+PrintGCTimeStamps, -XX:+PrintGCDateStamps, -Xloggc:gc.log For checking metaspace, -verbose:class or -XX:+TraceClassLoading , -XX:+TraceClassUnloading We can analyse logs through manual inspection, GCViewer, GCHisto, gceasy.io Heap dumps help determine unexpected memory growth and memory leaks. We can take heap dumps in follwing ways: jcmd pid GC.heap_dump heapdump.dmp jmap -dump:format=b,file=snapshot.jmap pid JConsole or Java Mission Control using MBean HotSpotDiagnostic JVM option heap dump on OOM error : -XX:+HeapDumpOnOutOfMemoryError . Frequent full GCs can delay collection of heap dump and restart of the process Eclipse Memory Analyzer Tool (MAT) shows leak suspects, histograms, unreachable objects, duplicate classes, reference chains to GC roots, allows using OQL to explore heap dumps. JOverFlow for JMC and Java VisualVM, YourKit (a commercial profiler) can all take heap dumps. Heap histograms - quick view of objects in heap Collect using: -XX:+PrintClassHistogram and SIGQUIT on Posix and SIGBREAK on Windows jcmd pid GC.class_histogram filename=histo jmap -histo pid core_file jhsdb jmap (Java 9) Java flight recordings - unexpected memory growth and memory leaks, GC events Enable Heap Statistics. Can introduce additional performance overhead To create a flight recording : -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=delay=20s,duration=60s,name=Rec,filename=lol.jfr,settings=profile Flight recordings can find out type of leaking objects but you need heap dumps to find out what is causing the objects to leak Finalizers Collect data using JConsole, jmap Analyse using Eclipse MAT / Visual VM using heap dumps Native Memory Native Memory Tracker output - tracks native memory used internally by JVM, not for external libraries. Start JVM with NativeMemoryTracking option pmap, libumem, valgrind, core files Conclusion In this series, we have taken a look at how the JVM manages memory and how the garbage collection process works. We have also gone through how to diagnose memory issues, which tools to use to collect and analyze diagnostic information and some JVM options which can affect application performance.\n","permalink":"https://rrampage.github.io/2018/05/16/jvm-primer-part-2-debugging-memory-issues/","summary":"\u003cp\u003eThis is part 2 of my series on JVM Memory management and debugging.\nRead part 1 \u003ca href=\"https://dev.to/rrampage/a-primer-on-jvm-memory-management-and-troubleshooting---1-12b6\"\u003ehere\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eIn this post, we will cover symptoms of memory issues for JVM-based applications, which tools we can use to diagnose them and how we can fix them.\u003c/p\u003e\n\u003ch2 id=\"symptoms\"\u003eSymptoms\u003c/h2\u003e\n\u003cp\u003eHere are a few symptoms of memory issues:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003ePoor application performance\u003c/li\u003e\n\u003cli\u003eAbnormal memory usage\u003c/li\u003e\n\u003cli\u003eOutOfMemory errors (OOME)\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch3 id=\"poor-application-performance\"\u003ePoor Application Performance\u003c/h3\u003e\n\u003col\u003e\n\u003cli\u003eApplication not performing to expected level\u003c/li\u003e\n\u003cli\u003eLong response times\u003c/li\u003e\n\u003cli\u003eDropping client requests\u003c/li\u003e\n\u003cli\u003eStuck threads\u003c/li\u003e\n\u003cli\u003eService unavailability\u003c/li\u003e\n\u003cli\u003eLarge gaps in timestamps in application logs\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch3 id=\"causes-of-memory-problems\"\u003eCauses of memory problems:\u003c/h3\u003e\n\u003col\u003e\n\u003cli\u003eMisconfigured memory\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eOld generation\u003c/strong\u003e memory space is sized smaller than live-set of objects. This triggers a major garbage collection (GC), resulting in larger pauses.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eCode cache\u003c/strong\u003e is smaller than generated compiled code footprint\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eYoung generation\u003c/strong\u003e is not sized appropriately leading to premature promotion of objects\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePermGen / Metaspace\u003c/strong\u003e not sized correctly leading to full GC\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eMemory leaks - Unintentional retention of objects in memory spaces\n\u003cul\u003e\n\u003cli\u003eUnintentional references to set of objects in heap\u003c/li\u003e\n\u003cli\u003eNot dereferencing classloader instances appropriateky\u003c/li\u003e\n\u003cli\u003eNot releasing native resources appropriately\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eExcessive use of \u003cstrong\u003efinalizers\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eObjects with finalizers may delay their own GC\u003c/li\u003e\n\u003cli\u003eFinalizer thread needs to invoke finalize() method of the instances before reclaiming them\u003c/li\u003e\n\u003cli\u003eThere can only be 1 Finalizer thread. If it does not keep up with rate at which objects become available for finalization, JVM fails with OOME\u003c/li\u003e\n\u003cli\u003ePending finalizer objects are essentially accumulated garbage\u003c/li\u003e\n\u003cli\u003eFinalizers deprecated in Java 9\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eExplicit GC calls\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003e\u003ccode\u003eSystem.gc()\u003c/code\u003e\u003c/strong\u003e and diagnostic data collections can cause long pauses\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003e\u003ccode\u003e-XX:+DisableExplicitGC\u003c/code\u003e\u003c/strong\u003e can disable System.gc() calls\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003e\u003ccode\u003e-XX:+PrintClassHistogram\u003c/code\u003e\u003c/strong\u003e also calls an explicit GC when receiving \u003ccode\u003ekill -3\u003c/code\u003e signal\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch3 id=\"outofmemoryerror\"\u003eOutOfMemoryError\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eHierarchy : \u003ccode\u003eThrowable\u003c/code\u003e -\u0026gt; \u003ccode\u003eError\u003c/code\u003e -\u0026gt; \u003ccode\u003eVirtualMachineError\u003c/code\u003e -\u0026gt; \u003ccode\u003eOutOfMemoryError\u003c/code\u003e (Unchecked exception)\u003c/li\u003e\n\u003cli\u003eThrown when JVM runs out of space in various memory spaces or cannot proceed further with process execution. Some of the possibilities:\n\u003cul\u003e\n\u003cli\u003eHeap space full\n\u003cul\u003e\n\u003cli\u003eJVM already invoked full GC but could not free up space\u003c/li\u003e\n\u003cli\u003eHeap may be sized smaller than app footprint or app is unnecessarily holding on to some set of objects in heap\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eGC overhead limit exceeded\n\u003cul\u003e\n\u003cli\u003eToo many GCs with very less space claimed\u003c/li\u003e\n\u003cli\u003eApplication threads not getting any CPU cycles\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eRequested array size exceeds VM limit\u003c/li\u003e\n\u003cli\u003ePermGen space / Metaspace / compressed class space\n\u003cul\u003e\n\u003cli\u003eFull GC invoked but unable to free space in Metaspace and application is attempting to load more classes\u003c/li\u003e\n\u003cli\u003eMetaspace by default \u0026ldquo;unlimited\u0026rdquo; but can be controlled by MaxMetaspaceSize. By default, 1 GB reserved for compressed class space\u003c/li\u003e\n\u003cli\u003eMake sure that -Xnoclassgc is not in use as it prevents unloading of classes\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eNative memory - out of swap space / stack trace with native method\n\u003cul\u003e\n\u003cli\u003eNative space used for Java thread stacks, loaded jars, zips, native libraries, native resources like files; mem allocated from native code\u003c/li\u003e\n\u003cli\u003eUnable to allocate more native memory or to create new threads or native memory leaks\u003c/li\u003e\n\u003cli\u003eRunning 32 bit JVM on 64 bit machine puts 4 GB limit on process size\u003c/li\u003e\n\u003cli\u003ePosition of Java heap can put a cap on max size of native heap. Can be controlled by option -XX:HeapBaseMinAddress=n to specify address native heap should be based at\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"codecache-warnings\"\u003eCodeCache warnings\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003ewarning message printed by JVM saying CodeCache full, compiler has been disabled.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eNo OOME when code cache is full\u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003eEmergency cleanup undertaken by Sweeper. This may discard compiled code and JIT may need to perform optimizations again\u003c/li\u003e\n\u003cli\u003eEnsure appropriate size of CC using \u003cstrong\u003e\u003ccode\u003eReservedCodeCacheSize\u003c/code\u003e\u003c/strong\u003e option\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"direct-buffer-memory\"\u003eDirect Buffer Memory\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003e\u003ccode\u003eByteBuffer.allocateDirect(N)\u003c/code\u003e\u003c/strong\u003e : Direct buffers which are garbage collected using phantom references and a reference queue\u003c/li\u003e\n\u003cli\u003eUnlimited memory by default but can be controlled by -XX:MaxDirectMemorySize=n\u003c/li\u003e\n\u003cli\u003eUsed by \u003cstrong\u003eJava NIO\u003c/strong\u003e. Heap ByteBuffer for I/O uses temporary direct ByteBuffer\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"diagnostic-data-data-collection-and-analysis-tools\"\u003eDiagnostic Data, Data Collection and Analysis Tools\u003c/h2\u003e\n\u003ch3 id=\"troubleshooting-memory-leaks\"\u003eTroubleshooting Memory leaks\u003c/h3\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003e\u003cstrong\u003eConfirm memory leak\u003c/strong\u003e\u003c/p\u003e","title":"JVM Primer Part 2 - Debugging memory issues"},{"content":"This series is a summary of Oracle\u0026rsquo;s JVM troubleshooting course which gives an overview on JVM memory management, Hotspot VM\u0026rsquo;s garbage collection options, various memory errors and how to troubleshoot them.\nIn this post (part 1), we will have a look at how JVM manages memory and its different garbage collectors.\nYou can find Part 2 here\nJVM Memory Management Overview The JVM provides automatic memory management to free the programmer from manually managing memory. New objects are allocated on heap memory. A root set consists of pointers to external memory, static variables, threads, JNI references and internal JVM structures. Objects directly reachable from the root set must be kept in heap. Objects reachable from any of the reachable objects must also be in heap. This group of objects are the only ones which can be used by a program. The unreachable objects (garbage) are removed using a process called garbage collection (GC). Reachable objects are compacted i.e moved to contiguous space in heap. This is important as otherwise, the heap will become fragmented.\nGenerational GC and Memory Spaces in Hotspot When the JVM starts, it requests some memory from the OS. This memory is divided into various spaces.\nMemory spaces in JVM prior to JDK 8\nSeparate pools hold objects of diff age ranges JVM\u0026rsquo;s GC is generational and is based on the hypothesis that: Most objects die young Few references from older to younger objects There are 2 generations of objects: young : small and collected frequently (minor collection). Objects which survive a threshold number of GCs move to old generation. old : large, collected infrequently (major collection = Full GC) Prior to JDK 8, there was also a permanent generation which was for storing class representations and metadata, interned strings and class statics. This was replaced by meta-space in JDK 8 and later. Meta-space is allocated in native memory. It is managed through JVM options MetaspaceSize for initial size and MaxMetaspaceSize for maximum. If UseCompressedClassPointers is enabled, 2 areas of memory are used for storing classes and their metadata - meta-space and compressed class space. 64 bit class pointers represented with 32 bit offsets to save space. Class metadata is referenced by 32 bit offsets stored in compressed class space. By default, compressed class space is 1 GB. Code cache is used to store compiled code generated by JIT (Just in time optimizer), allocated out of native memory and managed by Code Cache Sweeper Garbage Collectors in Hotspot JVM The JVM has different garbage collection methods for different generations of objects. Some of them are described below:\nYoung generation collection Serial - Stop-the-world (STW), copying collector, single threaded ParNew - STW, copying collector, multiple GC threads Parallel Scavenge - STW, copying collector, multiple GC threads Old generation collection Serial Old - STW, mark-sweep-compact collector, single threaded CMS - Mostly concurrent, low pause Parallel Old - compacting collector, multiple GC threads G1 : designed for large heaps and offers predictable short pauses. Has different memory layout for generations Same collector for all generations GC options for JDK These are the option flags passed to JVM for specifying which GC to use:\nUseSerialGC : Serial + SerialOld UseParNewGC : ParNew + SerialOld . In JDK 9, uses CMS for old gen UseConcMarkSweepGC : ParNew + CMS + SerialOld CMS used most of time to collect old generation. SerialOld used when concurrent mode failure occurs. CMS performs most work in concurrent with application threads. No heap compaction leads to fragmentation. Has floating garbage and requires larger heap sizes. Free space maintained as linked lists. Allocation expensive compared to bump-the-pointer allocations. Additional overhead on young collections Deprecated in JDK 9 UseParallelGC : Parallel Scavenge + Parallel Old. Maximizes throughput. Default GC till JDK 9 UseG1GC - G1 for both generations Server style GC for multi-core machines with large memory Low GC pauses with high probability while trying for high throughput Compacting collector. Low pauses without fragmentation Better GC ergonomics. Parallel threads and some tasks are concurrent with application threads Available since JDK 7u4 and default in JDK 9 For more detailed information on tuning the garbage collector, read the official GC Tuning Guide\nMinor GC or How young generation is collected: When Eden space in young gen is full, reachable objects are marked and moved to the ToSurvivorSpace Objects in FromSurvivor space that are reachable are moved to ToSurvivorSpace Objects in FromSurvivor space that have crossed the threshold are promoted to the old generation Eden becomes empty and is ready for new allocations To and From Survivor Spaces are switched Notes on Mark-Sweep-Compact collector (Serial Old): Mark phase : marks all live objects Sweep phase : sweeps over heap identifying garbage Slide phase : GC performs a sliding compaction by sliding live objects to the start of the heap ","permalink":"https://rrampage.github.io/2018/05/15/a-primer-on-jvm-memory-management-and-troubleshooting-1/","summary":"\u003cp\u003eThis series is a summary of Oracle\u0026rsquo;s \u003ca href=\"https://apexapps.oracle.com/pls/apex/f?p=44785:149:105166169953256::NO:RP,149:P149_EVENT_ID:5552\"\u003eJVM troubleshooting course\u003c/a\u003e which gives an overview on JVM memory management, Hotspot VM\u0026rsquo;s garbage collection options, various memory errors and how to troubleshoot them.\u003c/p\u003e\n\u003cp\u003eIn this post (part 1), we will have a look at how JVM manages memory and its different garbage collectors.\u003c/p\u003e\n\u003cp\u003eYou can find Part 2 \u003ca href=\"https://dev.to/rrampage/jvm-primer-part-2---debugging-performance-issues-1od\"\u003ehere\u003c/a\u003e\u003c/p\u003e\n\u003ch3 id=\"jvm-memory-management-overview\"\u003eJVM Memory Management Overview\u003c/h3\u003e\n\u003cp\u003eThe JVM provides automatic memory management to free the programmer from manually managing memory. New objects are allocated on heap memory. A \u003cstrong\u003eroot set\u003c/strong\u003e consists of pointers to external memory, static variables, threads, \u003ca href=\"https://en.wikipedia.org/wiki/Java_Native_Interface\"\u003eJNI\u003c/a\u003e references and internal JVM structures. Objects directly reachable from the root set must be kept in heap. Objects reachable from any of the reachable objects must also be in heap. This group of objects are the only ones which can be used by a program. The unreachable objects (garbage) are removed using a process called \u003cstrong\u003egarbage collection (GC)\u003c/strong\u003e. Reachable objects are \u003cstrong\u003ecompacted\u003c/strong\u003e i.e moved to contiguous space in heap. This is important as otherwise, the heap will become \u003ca href=\"https://en.wikipedia.org/wiki/Fragmentation_(computing)\"\u003efragmented\u003c/a\u003e.\u003c/p\u003e","title":"A Primer on JVM Memory Management and Troubleshooting - 1"},{"content":"Snake case and Camel case are conventions of naming variables, functions and classes. Most teams and projects prescribe a particular case in their style guides.\nExamples of camel case:\nMyClass MyClassFactory MyClassFactoryBuilder MyClassFactoryBuilderImpl myInstance myInstance2 abc patternMatcher Examples of snake case:\nadd matrix_add diagonal_matrix_add pseudo_inverse If we want to convert back and forth between these cases, we must look for the points of interest - the word boundaries. Camel case boundaries have the first letter capitalized while the snake case word boundaries have an _.\nSnake case to camel case Here is a regular expression for finding out the _ and the first letter in the next word:\n(.*?)_([a-zA-Z]) This regex has 2 parts:\n(.*?) finds everything upto the _.\nThe \u0026lsquo;.\u0026rsquo; means any character. \u0026lsquo;*\u0026rsquo; stands for match 0 or more instances \u0026lsquo;?\u0026rsquo; stands for non-greedy match. We must use \u0026lsquo;?\u0026rsquo; in the pattern because the regex engine will try to match as much as possible by default. So, if we use just (.*), the whole word will be consumed and nothing will be left for the rest of the pattern. \u0026lsquo;()\u0026rsquo; stand for a group. A group is a way of saving a part of the match for later. Together, they mean that find all characters upto the first \u0026lsquo;_\u0026rsquo; and capture them in a group. ([a-zA-Z]) finds the first alphabet after the _. We need this to convert to upper case for Camel case.\nThe Python code below converts words which are in snake case to camel case:\nimport re REG = r\u0026#34;(.*?)_([a-zA-Z])\u0026#34; def camel(match): return match.group(1) + match.group(2).upper() def camel_upper(match): return match.group(1)[0].upper() + match.group(1)[1:] + match.group(2).upper() words = \u0026#34;\u0026#34;\u0026#34;add matrix_add diagonal_matrix_add pseudo_inverse\u0026#34;\u0026#34;\u0026#34;.splitlines() results = [re.sub(REG, camel, w, 0) for w in words] print(results) # Output: # [\u0026#39;add\u0026#39;, \u0026#39;matrixAdd\u0026#39;, \u0026#39;diagonalMatrixAdd\u0026#39;, \u0026#39;pseudoInverse\u0026#39;] We use the regex we constructed earlier and the re.sub method to substitute our matched words. We pass a method called camel as an argument. This method allows us to change the case of the letter in the second group and keep the first group unchanged. Notice that the first letter can be either lower or upper depending on whether it is a variable or method (lower) or a class (upper). The camel_upper method can be used for class names.\nCamel case to snake case Similarly, for converting from camel to snake case, the regex is:\n(.+?)([A-Z]) And the Python code :\nimport re REG = r\u0026#34;(.+?)([A-Z])\u0026#34; def snake(match): return match.group(1).lower() + \u0026#34;_\u0026#34; + match.group(2).lower() words = \u0026#34;\u0026#34;\u0026#34;MyClass MyClassFactory MyClassFactoryBuilder MyClassFactoryBuilderImpl myInstance myInstance2 abc patternMatcher\u0026#34;\u0026#34;\u0026#34;.splitlines() results = [re.sub(REG, snake, w, 0) for w in words] print(results) # Output # [\u0026#39;my_class\u0026#39;, \u0026#39;my_class_factory\u0026#39;, \u0026#39;my_class_factory_builder\u0026#39;, \u0026#39;my_class_factory_builder_impl\u0026#39;, \u0026#39;my_instance\u0026#39;, \u0026#39;my_instance2\u0026#39;, \u0026#39;abc\u0026#39;, \u0026#39;pattern_matcher\u0026#39;] ","permalink":"https://rrampage.github.io/2018/05/09/snake-case-to-camel-case-and-back-using-regular-expressions-and-python/","summary":"\u003cp\u003e\u003ca href=\"https://en.wikipedia.org/wiki/Snake_case\"\u003eSnake case\u003c/a\u003e and \u003ca href=\"https://en.wikipedia.org/wiki/Camel_case\"\u003eCamel case\u003c/a\u003e are conventions of naming variables, functions and classes. Most teams and projects prescribe a particular case in their style guides.\u003c/p\u003e\n\u003cp\u003eExamples of camel case:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eMyClass\nMyClassFactory\nMyClassFactoryBuilder\nMyClassFactoryBuilderImpl\nmyInstance\nmyInstance2\nabc\npatternMatcher\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eExamples of snake case:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eadd\nmatrix_add\ndiagonal_matrix_add\npseudo_inverse\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eIf we want to convert back and forth between these cases, we must look for the points of interest - the word boundaries. Camel case boundaries have the first letter capitalized while the snake case word boundaries have an \u003cstrong\u003e\u003ccode\u003e_\u003c/code\u003e\u003c/strong\u003e.\u003c/p\u003e\n\u003ch3 id=\"snake-case-to-camel-case\"\u003eSnake case to camel case\u003c/h3\u003e\n\u003cp\u003eHere is a regular expression for finding out the \u003cstrong\u003e\u003ccode\u003e_\u003c/code\u003e\u003c/strong\u003e and the first letter in the next word:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode class=\"language-regex\" data-lang=\"regex\"\u003e(.*?)_([a-zA-Z])\n\u003c/code\u003e\u003c/pre\u003e\u003cp\u003eThis regex has 2 parts:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003e\u003cstrong\u003e\u003ccode\u003e(.*?)\u003c/code\u003e\u003c/strong\u003e finds everything upto the \u003cstrong\u003e\u003ccode\u003e_\u003c/code\u003e\u003c/strong\u003e.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThe \u0026lsquo;.\u0026rsquo; means any character.\u003c/li\u003e\n\u003cli\u003e\u0026lsquo;*\u0026rsquo; stands for match 0 or more instances\u003c/li\u003e\n\u003cli\u003e\u0026lsquo;?\u0026rsquo; stands for non-greedy match. We must use \u0026lsquo;?\u0026rsquo; in the pattern because the regex engine will try to match as much as possible by default. So, if we use just \u003cstrong\u003e\u003ccode\u003e(.*)\u003c/code\u003e\u003c/strong\u003e, the whole word will be consumed and nothing will be left for the rest of the pattern.\u003c/li\u003e\n\u003cli\u003e\u0026lsquo;()\u0026rsquo; stand for a group. A group is a way of saving a part of the match for later.\u003c/li\u003e\n\u003cli\u003eTogether, they mean that find all characters upto the first \u0026lsquo;_\u0026rsquo; and capture them in a group.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003e\u003cstrong\u003e\u003ccode\u003e([a-zA-Z])\u003c/code\u003e\u003c/strong\u003e finds the first alphabet after the \u003cstrong\u003e\u003ccode\u003e_\u003c/code\u003e\u003c/strong\u003e. We need this to convert to upper case for Camel case.\u003c/p\u003e","title":"Snake case to camel case and back using regular expressions and Python"},{"content":"Background Today, I resumed a rust project of mine after a long time. In order to check my last working code, I ran cargo run. But it refused to run with error message:\n....\u0026#34;-Wl,-Bdynamic\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;dl\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;rt\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;pthread\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;pthread\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;gcc_s\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;c\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;m\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;rt\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;pthread\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;util\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;util\u0026#34; = note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory collect2: error: ld returned 1 exit status Linker Issues! ld is used to link libraries. So, the error was not in the code but in the linking phase. I googled the error and the solution was to have build-essential package installed. But the package was already installed on my machine (it is one of the first packages I install on any development machine).\nSome more googling revealed that cargo uses the system cc linker to link to the C runtime. Running which cc gave me $HOME/anaconda3/bin/cc. This cc is part of my Anaconda root environment. (Anaconda is a package manager for scientific computing packages. It is a convenient way for installing multiple versions of packages in different environments).\nOn Linux, the linker knows where to find the required libraries using the shared library cache. I ran ldconfig -v to refresh it and then try again. Same error!\nIt is possible to explicitly list directories to include using LD_LIBRARY_PATH environment variable. I tried setting the LD_LIBRARY_PATH to point to the required directory and then run cargo as LD_LIBRARY_PATH=DIR cargo build -v. But it gave the same error.\nI thought that cargo must be setting the linker value somewhere, so instead let me try directly compiling with rustc. Even that gave the same error. With this, I eliminated the possibility of some environment variable only affecting cargo.\nMore googling! Further searching for the error with \u0026ldquo;rust\u0026rdquo; added showed me results of people having trouble cross-compiling. From this, I learned that cargo has different targets i.e different instruction sets (e,g x86-64, arm, x86, mips), different OSs (e.g linux, windows, freebsd) and different C runtimes (e.g glibc, musl, msvc). The Rust documentation on cargo mentioned that this is called a target triple. The Cargo book mentioned that you can direct cargo to explicitly use a particular linker using RUSTFLAGS environment variable.\nSince I am only building for my machine, I had to find out the exact value of the target. Rust gives an exhaustive list of all supported targets by running rustc --print target-list. My target was x86_64-unknown-linux-gnu.\nIt is possible to pass a linker to cargo explicitly as RUSTFLAGS=\u0026quot;-C linker=x86_64-unknown-linux-gnu\u0026quot; cargo build -v. It worked!\nAll-time fix! I do not want to do this every time I run cargo. The Cargo book says that cargo uses a global config file in $HOME/.cargo/config.\nI added the following to the file:\n# Specify which linker to use for this target [target.x86_64-unknown-linux-gnu] linker = \u0026#34;x86_64-linux-gnu-gcc\u0026#34; Now, cargo build works without issues.\n","permalink":"https://rrampage.github.io/2018/05/09/cargo-refused-to-build-my-project-a-rust-debugging-story/","summary":"\u003ch3 id=\"background\"\u003eBackground\u003c/h3\u003e\n\u003cp\u003eToday, I resumed a \u003ccode\u003erust\u003c/code\u003e project of mine after a long time. In order to check my last working code, I ran \u003ccode\u003ecargo run\u003c/code\u003e. But it refused to run with error message:\u003c/p\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003e....\u0026#34;-Wl,-Bdynamic\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;dl\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;rt\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;pthread\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;pthread\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;gcc_s\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;c\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;m\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;rt\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;pthread\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;util\u0026#34; \u0026#34;-l\u0026#34; \u0026#34;util\u0026#34;\n= note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory\n          collect2: error: ld returned 1 exit status\n\u003c/code\u003e\u003c/pre\u003e\u003ch3 id=\"linker-issues\"\u003eLinker Issues!\u003c/h3\u003e\n\u003cp\u003e\u003ccode\u003eld\u003c/code\u003e is used to link libraries. So, the error was not in the code but in the linking phase. I googled the error and the solution was to have \u003ccode\u003ebuild-essential\u003c/code\u003e package installed. But the package was already installed on my machine (it is one of the first packages I install on any development machine).\u003c/p\u003e\n\u003cp\u003eSome more googling revealed that \u003ccode\u003ecargo\u003c/code\u003e uses the system \u003ccode\u003ecc\u003c/code\u003e linker to link to the C runtime. Running \u003ccode\u003ewhich cc\u003c/code\u003e gave me \u003ccode\u003e$HOME/anaconda3/bin/cc\u003c/code\u003e. This \u003ccode\u003ecc\u003c/code\u003e is part of my Anaconda root environment. (Anaconda is a package manager for scientific computing packages. It is a convenient way for installing multiple versions of packages in different environments).\u003c/p\u003e","title":"Cargo refused to build my project - A Rust debugging story!"},{"content":"Many times, smoothly running processes stop working mysteriously. You open the logs and see what is happening, only to find that even the logs have stopped updating. But the process itself is running. You SSH to the server and type cd TAB. Bash weeps \u0026ldquo;Unable to create temporary file\u0026rdquo;. The machine is out of disk space\u0026hellip;\nHere is a checklist to make disk space debugging easier, using standard Linux utilities so you can get started without having to install anything extra:\ndf -h command gives you an overview in a readable format about the number of disks mounted and their total and available capacities. To get an idea of which folders/directories are eating up the maximum space, try out du -ch / | sort -h | tail -n 30. This gives you the 30 most space consuming directories. If you already know which directories generate maximum disk output e.g logs and temp files, you can replace the \u0026lsquo;/\u0026rsquo; with your directory (DIR) and run the command as du -ch DIR | sort -h | tail -n 30 Now that we have identified the directories with maximum space consumed, we may need to delete some files and get our process going again. The rm command is your friend here. You can delete old logs and temporary files to free up space. Many times, the culprit is a single large file which is already in use by a program e.g catalina.out by Apache Tomcat. If you want to free up space without shutting down the process, the truncate command will help you out. Example: truncate -s0 BIG_LOG.log. This will truncate the file to 0 bytes and still allow the other process to use it without issues (standard Unix permissions apply) Sometimes, you delete files and still, the space does not seem to be recovered. This can be because some process is still holding on to the file descriptor of the deleted file. Once these processes are stopped, the space will be recovered. The lsof command will help you out here. It stands for list open files. You can find out which processes are using deleted files as follows: lsof | grep deleted | grep OLD_FILENAME. The lsof command gives you the process name and the process id so you can run kill on the process. If you do not know the name of the deleted file, you can still run lsof | grep deleted and see the output to check for any familiar file / process. Finally, keep in mind that disk space is one of the metrics you should monitor on your server. This checklist must be used in a pinch. If you find yourself constantly having disk space issues, the solution is to set up periodic deletion/rotation of old log files, alerts when the disk space reaches a particular threshold or to increase the disk size if your processes require a lot of disk space e.g Kafka, MySQL and other databases.\nLet me know if there are some other tools I am missing out on and your experiences dealing with disk space issues!\n","permalink":"https://rrampage.github.io/2018/05/04/disk-space-debugging-checklist/","summary":"\u003cp\u003eMany times, smoothly running processes stop working mysteriously. You open the logs and see what is happening, only to find that even the logs have stopped updating. But the process itself is running. You SSH to the server and type cd TAB. Bash weeps \u0026ldquo;Unable to create temporary file\u0026rdquo;. The machine is out of disk space\u0026hellip;\u003c/p\u003e\n\u003cp\u003eHere is a checklist to make disk space debugging easier, using standard Linux utilities so you can get started without having to install anything extra:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003e\u003ccode\u003edf -h\u003c/code\u003e\u003c/strong\u003e command gives you an overview in a readable format about the number of disks mounted and their total and available capacities.\u003c/li\u003e\n\u003cli\u003eTo get an idea of which folders/directories are eating up the maximum space, try out \u003cstrong\u003e\u003ccode\u003edu -ch / | sort -h | tail -n 30\u003c/code\u003e\u003c/strong\u003e. This gives you the 30 most space consuming directories. If you already know which directories generate maximum disk output e.g logs and temp files, you can replace the \u0026lsquo;/\u0026rsquo; with your directory (DIR) and run the command as \u003cstrong\u003e\u003ccode\u003edu -ch DIR | sort -h | tail -n 30\u003c/code\u003e\u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003eNow that we have identified the directories with maximum space consumed, we may need to delete some files and get our process going again. The \u003cstrong\u003e\u003ccode\u003erm\u003c/code\u003e\u003c/strong\u003e command is your friend here. You can delete old logs and temporary files to free up space.\u003c/li\u003e\n\u003cli\u003eMany times, the culprit is a single large file which is already in use by a program e.g \u003ccode\u003ecatalina.out\u003c/code\u003e by Apache Tomcat. If you want to free up space without shutting down the process, the \u003cstrong\u003e\u003ccode\u003etruncate\u003c/code\u003e\u003c/strong\u003e command will help you out. Example: \u003cstrong\u003e\u003ccode\u003etruncate -s0 BIG_LOG.log\u003c/code\u003e\u003c/strong\u003e. This will truncate the file to 0 bytes and still allow the other process to use it without issues (standard Unix permissions apply)\u003c/li\u003e\n\u003cli\u003eSometimes, you delete files and still, the space does not seem to be recovered. This can be because some process is still holding on to the file descriptor of the deleted file. Once these processes are stopped, the space will be recovered. The \u003cstrong\u003e\u003ccode\u003elsof\u003c/code\u003e\u003c/strong\u003e command will help you out here. It stands for \u003cem\u003elist open files\u003c/em\u003e. You can find out which processes are using deleted files as follows: \u003ccode\u003elsof | grep deleted | grep OLD_FILENAME\u003c/code\u003e. The lsof command gives you the process name and the process id so you can run \u003ccode\u003ekill\u003c/code\u003e on the process. If you do not know the name of the deleted file, you can still run \u003ccode\u003elsof | grep deleted\u003c/code\u003e and see the output to check for any familiar file / process.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eFinally, keep in mind that disk space is one of the metrics you should monitor on your server. This checklist must be used in a pinch. If you find yourself constantly having disk space issues, the solution is to set up periodic deletion/rotation of old log files, alerts when the disk space reaches a particular threshold or to increase the disk size if your processes require a lot of disk space e.g Kafka, MySQL and other databases.\u003c/p\u003e","title":"Disk Space Debugging Checklist"},{"content":"Markdown Basics You can write regular markdown here and Jekyll will automatically convert it to a nice webpage. I strongly encourage you to take 5 minutes to learn how to write in markdown - it\u0026rsquo;ll teach you how to transform regular text into bold/italics/headings/tables/etc.\nHere is some bold text\nHere is a secondary heading Here\u0026rsquo;s a useless table:\nNumber Next number Previous number Five Six Four Ten Eleven Nine Seven Eight Six Two Three One How about a yummy crepe?\nCode and Syntax highlighting The following are two code samples using syntax highlighting.\nThe following is a code sample using triple backticks ( ``` ) code fencing provided in Hugo. This is client side highlighting and does not require any special installation.\nvar num1, num2, sum num1 = prompt(\u0026#34;Enter first number\u0026#34;) num2 = prompt(\u0026#34;Enter second number\u0026#34;) sum = parseInt(num1) + parseInt(num2) // \u0026#34;+\u0026#34; means \u0026#34;add\u0026#34; alert(\u0026#34;Sum = \u0026#34; + sum) // \u0026#34;+\u0026#34; means combine into a string The following is a code sample using the \u0026ldquo;highlight\u0026rdquo; shortcode provided in Hugo. This is server side highlighting and requires Python and Pygments to be installed.\nvar num1, num2, sum num1 = prompt(\u0026#34;Enter first number\u0026#34;) num2 = prompt(\u0026#34;Enter second number\u0026#34;) sum = parseInt(num1) + parseInt(num2) // \u0026#34;+\u0026#34; means \u0026#34;add\u0026#34; alert(\u0026#34;Sum = \u0026#34; + sum) // \u0026#34;+\u0026#34; means combine into a string And here is the same code with line numbers:\n1 var num1, num2, sum 2 num1 = prompt(\u0026#34;Enter first number\u0026#34;) 3 num2 = prompt(\u0026#34;Enter second number\u0026#34;) 4 sum = parseInt(num1) + parseInt(num2) // \u0026#34;+\u0026#34; means \u0026#34;add\u0026#34; 5 alert(\u0026#34;Sum = \u0026#34; + sum) // \u0026#34;+\u0026#34; means combine into a string Math and KaTeX KaTeX can be used to generate complex math formulas server-side.\n$$ \\phi = \\frac{(1+\\sqrt{5})}{2} = 1.6180339887\\cdots $$\nAdditional details can be found on GitHub or on the Wiki.\nTo load KaTeX on markdown pages, add math: true in front-matter of post. Read this post for more information\nExample 1 If the text between $$ contains newlines it will rendered in display mode:\n$$ f(x) = \\int_{-\\infty}^\\infty\\hat f(\\xi)\\,e^{2 \\pi i \\xi x}\\,d\\xi $$ $$ f(x) = \\int_{-\\infty}^\\infty\\hat f(\\xi),e^{2 \\pi i \\xi x},d\\xi $$\nExample 2 $$ \\frac{1}{\\Bigl(\\sqrt{\\phi \\sqrt{5}}-\\phi\\Bigr) e^{\\frac25 \\pi}} = 1+\\frac{e^{-2\\pi}} {1+\\frac{e^{-4\\pi}} {1+\\frac{e^{-6\\pi}} {1+\\frac{e^{-8\\pi}} {1+\\cdots} } } } $$ ​​$$ \\frac{1}{\\Bigl(\\sqrt{\\phi \\sqrt{5}}-\\phi\\Bigr) e^{\\frac25 \\pi}} = 1+\\frac{e^{-2\\pi}} {1+\\frac{e^{-4\\pi}} {1+\\frac{e^{-6\\pi}} {1+\\frac{e^{-8\\pi}} {1+\\cdots} } } } $$ ​​\nExample 3 $$ 1 + \\frac{q^2}{(1-q)}+\\frac{q^6}{(1-q)(1-q^2)}+\\cdots = \\prod_{j=0}^{\\infty}\\frac{1}{(1-q^{5j+2})(1-q^{5j+3})}, \\quad\\quad \\text{for }\\lvert q\\rvert\u0026lt;1. $$ $$ 1 + \\frac{q^2}{(1-q)}+\\frac{q^6}{(1-q)(1-q^2)}+\\cdots = \\prod_{j=0}^{\\infty}\\frac{1}{(1-q^{5j+2})(1-q^{5j+3})}, \\quad\\quad \\text{for }\\lvert q\\rvert\u0026lt;1. $$\n","permalink":"https://rrampage.github.io/2017/08/20/markdown-and-some-hugo-tutorial/","summary":"\u003ch1 id=\"markdown-basics\"\u003eMarkdown Basics\u003c/h1\u003e\n\u003cp\u003eYou can write regular \u003ca href=\"http://markdowntutorial.com/\"\u003emarkdown\u003c/a\u003e here and Jekyll will automatically convert it to a nice webpage.  I strongly encourage you to \u003ca href=\"http://markdowntutorial.com/\"\u003etake 5 minutes to learn how to write in markdown\u003c/a\u003e - it\u0026rsquo;ll teach you how to transform regular text into bold/italics/headings/tables/etc.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eHere is some bold text\u003c/strong\u003e\u003c/p\u003e\n\u003ch2 id=\"here-is-a-secondary-heading\"\u003eHere is a secondary heading\u003c/h2\u003e\n\u003cp\u003eHere\u0026rsquo;s a useless table:\u003c/p\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth style=\"text-align: left\"\u003eNumber\u003c/th\u003e\n          \u003cth style=\"text-align: left\"\u003eNext number\u003c/th\u003e\n          \u003cth style=\"text-align: left\"\u003ePrevious number\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd style=\"text-align: left\"\u003eFive\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eSix\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eFour\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd style=\"text-align: left\"\u003eTen\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eEleven\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eNine\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd style=\"text-align: left\"\u003eSeven\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eEight\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eSix\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd style=\"text-align: left\"\u003eTwo\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eThree\u003c/td\u003e\n          \u003ctd style=\"text-align: left\"\u003eOne\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eHow about a yummy crepe?\u003c/p\u003e\n\u003cp\u003e\u003cimg loading=\"lazy\" src=\"http://s3-media3.fl.yelpcdn.com/bphoto/cQ1Yoa75m2yUFFbY2xwuqw/348s.jpg\" alt=\"Crepe\"  /\u003e\n\u003c/p\u003e\n\u003ch1 id=\"code-and-syntax-highlighting\"\u003eCode and Syntax highlighting\u003c/h1\u003e\n\u003cp\u003eThe following are two code samples using syntax highlighting.\u003c/p\u003e","title":"Markdown (and some Hugo) Tutorial"},{"content":"Hello World!\n","permalink":"https://rrampage.github.io/2014/03/03/hello-world/","summary":"\u003cp\u003e\u003cstrong\u003eHello World!\u003c/strong\u003e\u003c/p\u003e","title":"Hello World"},{"content":"About this blog: the delights and frustrations of programming new things I learn/stumble across while coding interesting concepts/resources I come across trying to explain a few topics About Me I am a software developer. I work on mostly backend stuff. I blog occasionally here or crosspost to dev.to.\n💬 Ask me about \u0026hellip; Bash, sed, awk, GNU Make, jq Linux Postgres/MySQL/SQLite Java/Kotlin, JOOQ, Dropwizard Solr \u0026amp; Elasticsearch ArangoDB Nginx \u0026amp; Openresty State machines, Regexes, State charts 📫 How to reach me: Mastodon LinkedIn Bluesky Software powering this blog: Hugo Modified version of PaperMod theme Github pages ","permalink":"https://rrampage.github.io/about/","summary":"\u003ch4 id=\"about-this-blog\"\u003eAbout this blog:\u003c/h4\u003e\n\u003cul\u003e\n\u003cli\u003ethe delights and frustrations of programming\u003c/li\u003e\n\u003cli\u003enew things I learn/stumble across while coding\u003c/li\u003e\n\u003cli\u003einteresting concepts/resources I come across\u003c/li\u003e\n\u003cli\u003etrying to explain a few topics\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch4 id=\"about-me\"\u003eAbout Me\u003c/h4\u003e\n\u003cp\u003eI am a software developer. I work on mostly backend stuff.\nI blog occasionally \u003ca href=\"https://rrampage.github.io\"\u003ehere\u003c/a\u003e or crosspost to \u003ca href=\"https://dev.to/rrampage\"\u003edev.to\u003c/a\u003e.\u003c/p\u003e\n\u003ch4 id=\"-ask-me-about-\"\u003e💬 Ask me about \u0026hellip;\u003c/h4\u003e\n\u003cul\u003e\n\u003cli\u003eBash, sed, awk, GNU Make, jq\u003c/li\u003e\n\u003cli\u003eLinux\u003c/li\u003e\n\u003cli\u003ePostgres/MySQL/SQLite\u003c/li\u003e\n\u003cli\u003eJava/Kotlin, JOOQ, Dropwizard\u003c/li\u003e\n\u003cli\u003eSolr \u0026amp; Elasticsearch\u003c/li\u003e\n\u003cli\u003eArangoDB\u003c/li\u003e\n\u003cli\u003eNginx \u0026amp; Openresty\u003c/li\u003e\n\u003cli\u003eState machines, Regexes, State charts\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch4 id=\"-how-to-reach-me\"\u003e📫 How to reach me:\u003c/h4\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://hachyderm.io/@bit\"\u003eMastodon\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://in.linkedin.com/in/raunak1\"\u003eLinkedIn\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://bsky.app/profile/rrampage.bsky.social\"\u003eBluesky\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch4 id=\"software-powering-this-blog\"\u003eSoftware powering this blog:\u003c/h4\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://gohugo.io/\"\u003eHugo\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003eModified version of \u003ca href=\"https://github.com/adityatelange/hugo-PaperMod/\"\u003ePaperMod\u003c/a\u003e theme\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/rrampage/rrampage.github.io\"\u003eGithub pages\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e","title":"About"},{"content":"","permalink":"https://rrampage.github.io/misc/apl-scrapbook/","summary":"","title":"APL Scrapbook"}]