Commit caccd81c3e
Unsigned
Layout: unified · split
.gitignore added +2
| @@ -0,0 +1,2 @@ | ||
| 1 | public/ | |
| 2 | sitemap.org | |
README.org added +50
| @@ -0,0 +1,50 @@ | ||
| 1 | #+title: README | |
| 2 | ||
| 3 | * Overview | |
| 4 | ||
| 5 | This project uses [[https://orgmode.org/][org-mode]] to structure all documents, including the index page | |
| 6 | and sub-pages such as blog posts. | |
| 7 | ||
| 8 | * Configuration | |
| 9 | ||
| 10 | If you want to replicate this project structure, you'll need to customize your | |
| 11 | =.emacs= file appropriately. | |
| 12 | ||
| 13 | My =.emacs= file below show a handful of project-related settings | |
| 14 | ||
| 15 | #+BEGIN_SRC lisp | |
| 16 | ;; Project publishing settings | |
| 17 | (setq org-publish-project-alist | |
| 18 | '(("cleberg.net" | |
| 19 | :base-directory "~/Source/cleberg.net/" | |
| 20 | :publishing-function org-html-publish-to-html | |
| 21 | :publishing-directory "~/Source/cleberg.net/public/" | |
| 22 | :auto-sitemap t | |
| 23 | :recursive t | |
| 24 | :section-numbers nil | |
| 25 | :with-toc nil | |
| 26 | :with-author nil | |
| 27 | :html-validation-link nil | |
| 28 | :with-tags t | |
| 29 | :html-head "<link rel=stylesheet href=/static/styles.css>" | |
| 30 | :html-preamble "<nav><a href=/>Home</a><a href=/blog/>Blog</a><a href=/notes/>Notes</a><a href=/services/>Services</a></nav>") | |
| 31 | ||
| 32 | ("static" | |
| 33 | :base-directory "~/Source/cleberg.net/static/" | |
| 34 | :base-extension "css\\|el" | |
| 35 | :publishing-directory "~/Source/cleberg.net/public/static/" | |
| 36 | :publishing-function org-publish-attachment) | |
| 37 | ("website" :components ("cleberg.net" "static")))) | |
| 38 | #+END_SRC | |
| 39 | ||
| 40 | If you're within emacs while changing =.emacs=, you'll need to reload the | |
| 41 | configuration with =M-x load-file= and hit enter to reload the current file | |
| 42 | (=.emacs=). | |
| 43 | ||
| 44 | * Building | |
| 45 | ||
| 46 | When ready to publish, open any of the =.org= files in the project and execute | |
| 47 | =M-x org-publish-current-project=. | |
| 48 | ||
| 49 | If you want to force emacs to re-publish all files, including files without | |
| 50 | changes, execute =C-u M-x org-publish-current-project=. | |
blog/2018-11-28-cpp-compiler.org added +127
| @@ -0,0 +1,127 @@ | ||
| 1 | #+date:2018-11-28 | |
| 2 | #+title: The C++ Compiler | |
| 3 | ||
| 4 | * A Brief Introduction | |
| 5 | ||
| 6 | [[https://en.wikipedia.org/wiki/C%2B%2B][C++]] is a general-purpose programming language with object-oriented, generic, and | |
| 7 | functional features in addition to facilities for low-level memory manipulation. | |
| 8 | ||
| 9 | The source code, shown in the snippet below, must be compiled before it can be | |
| 10 | executed. There are many steps and intricacies to the compilation process, and | |
| 11 | this post was a personal exercise to learn and remember as much information as I | |
| 12 | can. | |
| 13 | ||
| 14 | #+BEGIN_SRC cpp | |
| 15 | #include <iostream> | |
| 16 | ||
| 17 | int main() | |
| 18 | { | |
| 19 | std::cout << "Hello, world!\n"; | |
| 20 | } | |
| 21 | #+END_SRC | |
| 22 | ||
| 23 | * Compilation Process | |
| 24 | ||
| 25 | ** An Overview | |
| 26 | ||
| 27 | Compiling C++ projects is a frustrating task most days. Seemingly nonexistent | |
| 28 | errors keeping your program from successfully compiling can be annoying | |
| 29 | (especially since you know you wrote it perfectly the first time, right?). | |
| 30 | ||
| 31 | I'm learning more and more about C++ these days and decided to write this | |
| 32 | concept down so that I can cement it even further in my own head. However, C++ | |
| 33 | is not the only compiled language. Check out [[https://en.wikipedia.org/wiki/Compiled_language][the Wikipedia entry for compiled | |
| 34 | languages]] for more examples of compiled languages. | |
| 35 | ||
| 36 | I'll start with a wonderful, graphical way to conceptualize the C++ compiler. | |
| 37 | View [[https://web.archive.org/web/20190419035048/http://faculty.cs.niu.edu/~mcmahon/CS241/Notes/compile.html][The C++ Compilation Process]] by Kurt MacMahon, an NIU professor, to see the | |
| 38 | graphic and an explanation. The goal of the compilation process is to take the | |
| 39 | C++ code and produce a shared library, dynamic library, or an executable file. | |
| 40 | ||
| 41 | ** Compilation Phases | |
| 42 | ||
| 43 | Let's break down the compilation process. There are four major steps to | |
| 44 | compiling C++ code. | |
| 45 | ||
| 46 | *** Step 1 | |
| 47 | ||
| 48 | The first step is to expand the source code file to meet all dependencies. The | |
| 49 | C++ preprocessor includes the code from all the header files, such as | |
| 50 | =#include <iostream>=. Now, what does that mean? The previous example includes | |
| 51 | the =iostream= header. This tells the computer that you want to use the | |
| 52 | =iostream= standard library, which contains classes and functions written in the | |
| 53 | core language. This specific header allows you to manipulate input/output | |
| 54 | streams. After all this, you'll end up which a temporary file that contains the | |
| 55 | expanded source code. | |
| 56 | ||
| 57 | In the example of the C++ code above, the =iostream= class would be included | |
| 58 | in the expanded code. | |
| 59 | ||
| 60 | *** Step 2 | |
| 61 | ||
| 62 | After the code is expanded, the compiler comes into play. The compiler takes the | |
| 63 | C++ code and converts this code into the assembly language, understood by the | |
| 64 | platform. You can see this in action if you head over to the [[https://godbolt.org][GodBolt Compiler | |
| 65 | Explorer]], which shows C++ being converted into assembly dynamically. | |
| 66 | ||
| 67 | For example, the =Hello, world!= code snippet above compiles into the following | |
| 68 | assembly code: | |
| 69 | ||
| 70 | #+BEGIN_SRC asm | |
| 71 | .LC0: | |
| 72 | .string "Hello, world!\n" | |
| 73 | main: | |
| 74 | push rbp | |
| 75 | mov rbp, rsp | |
| 76 | mov esi, OFFSET FLAT:.LC0 | |
| 77 | mov edi, OFFSET FLAT:_ZSt4cout | |
| 78 | call std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*) | |
| 79 | mov eax, 0 | |
| 80 | pop rbp | |
| 81 | ret | |
| 82 | __static_initialization_and_destruction_0(int, int): | |
| 83 | push rbp | |
| 84 | mov rbp, rsp | |
| 85 | sub rsp, 16 | |
| 86 | mov DWORD PTR [rbp-4], edi | |
| 87 | mov DWORD PTR [rbp-8], esi | |
| 88 | cmp DWORD PTR [rbp-4], 1 | |
| 89 | jne .L5 | |
| 90 | cmp DWORD PTR [rbp-8], 65535 | |
| 91 | jne .L5 | |
| 92 | mov edi, OFFSET FLAT:_ZStL8__ioinit | |
| 93 | call std::ios_base::Init::Init() [complete object constructor] | |
| 94 | mov edx, OFFSET FLAT:__dso_handle | |
| 95 | mov esi, OFFSET FLAT:_ZStL8__ioinit | |
| 96 | mov edi, OFFSET FLAT:_ZNSt8ios_base4InitD1Ev | |
| 97 | call __cxa_atexit | |
| 98 | .L5: | |
| 99 | nop | |
| 100 | leave | |
| 101 | ret | |
| 102 | _GLOBAL__sub_I_main: | |
| 103 | push rbp | |
| 104 | mov rbp, rsp | |
| 105 | mov esi, 65535 | |
| 106 | mov edi, 1 | |
| 107 | call __static_initialization_and_destruction_0(int, int) | |
| 108 | pop rbp | |
| 109 | ret | |
| 110 | #+END_SRC | |
| 111 | ||
| 112 | *** Step 3 | |
| 113 | ||
| 114 | Third, the assembly code generated by the compiler is assembled into the object | |
| 115 | code for the platform. Essentially, this is when the compiler takes the assembly | |
| 116 | code and assembles it into machine code in a binary format. After researching | |
| 117 | this online, I figured out that a lot of compilers will allow you to stop | |
| 118 | compilation at this step. This would be useful for compiling each source code | |
| 119 | file separately. This saves time later if a single file changes; only that file | |
| 120 | needs to be recompiled. | |
| 121 | ||
| 122 | *** Step 4 | |
| 123 | ||
| 124 | Finally, the object code file generated by the assembler is linked together with | |
| 125 | the object code files for any library functions used to produce a shared | |
| 126 | library, dynamic library, or an executable file. It replaces all references to | |
| 127 | undefined symbols with the correct addresses. | |
blog/2018-12-08-aes-encryption.org added +103
| @@ -0,0 +1,103 @@ | ||
| 1 | #+date: 2018-12-08 | |
| 2 | #+title: AES Encryption | |
| 3 | ||
| 4 | * Basic AES | |
| 5 | ||
| 6 | If you're not familiar with encryption techniques, [[https://en.wikipedia.org/wiki/Advanced_Encryption_Standard][AES]] is the *Advanced | |
| 7 | Encryption Standard*. This specification was established by the National | |
| 8 | Institute of Standards and Technology, sub-selected from the Rijndael family of | |
| 9 | ciphers (128, 192, and 256 bits) in 2001. Furthering its popularity and status, | |
| 10 | the US government chose AES as their default encryption method for top-secret | |
| 11 | data, removing the previous standard which had been in place since 1977. | |
| 12 | ||
| 13 | AES has proven to be an extremely safe encryption method, with 7-round and | |
| 14 | 8-round attacks making no material improvements since the release of this | |
| 15 | encryption standard almost two decades ago. | |
| 16 | ||
| 17 | #+BEGIN_QUOTE | |
| 18 | Though many papers have been published on the cryptanalysis of AES, the | |
| 19 | fastest single-key attacks on round-reduced AES variants [20, 33] so far are | |
| 20 | only slightly more powerful than those proposed 10 years ago [23,24]. | |
| 21 | - [[http://research.microsoft.com/en-us/projects/cryptanalysis/aesbc.pdf][Bogdonav, et al.]] | |
| 22 | #+END_QUOTE | |
| 23 | ||
| 24 | * How Secure is AES? | |
| 25 | ||
| 26 | In theory, AES-256 is non-crackable due to the massive number of combinations | |
| 27 | that can be produced. However, AES-128 is no longer recommended as a viable | |
| 28 | implementation to protect important data. | |
| 29 | ||
| 30 | A semi-short [[http://www.moserware.com/2009/09/stick-figure-guide-to-advanced.html][comic strip]] from Moserware quickly explains AES for the public to | |
| 31 | understand. Basically AES encrypts the data by obscuring the relationship | |
| 32 | between the data and the encrypted data. Additionally, this method spreads the | |
| 33 | message out. Lastly, the key produced by AES is the secret to decrypting | |
| 34 | it. Someone may know the method of AES, but without the key, they are powerless. | |
| 35 | ||
| 36 | To obscure and spread the data out, AES creates a substitution-permutation | |
| 37 | network. Wikipedia has a wonderful [[https://upload.wikimedia.org/wikipedia/commons/thumb/c/cd/SubstitutionPermutationNetwork2.png/468px-SubstitutionPermutationNetwork2.png][example of an SP network]] available. This | |
| 38 | network sends the data through a set of S boxes (using the unique key) to | |
| 39 | substitute the bits with another block of bits. Then, a P box will permutate, or | |
| 40 | rearrange, the bits. This is done over and over, with the key being derived from | |
| 41 | the last round. For AES, the key size specifies the number of transformation | |
| 42 | rounds: 10, 12, and 14 rounds for 128-bit, 192-bit, and 256-bit keys, | |
| 43 | respectively. | |
| 44 | ||
| 45 | * The Process | |
| 46 | ||
| 47 | 1. *KeyExpansion=: Using [[https://en.m.wikipedia.org/wiki/Advanced_Encryption_Standard][Rijndael's key schedule]], the keys are dynamically | |
| 48 | generated. | |
| 49 | 2. *AddRoundKey*: Each byte of the data is combined with this key using bitwise | |
| 50 | xor. | |
| 51 | 3. *SubBytes*: This is followed by the substitution of each byte of data. | |
| 52 | 4. *ShiftRows*: Then, the final three rows are shifted a certain number of | |
| 53 | steps, dictated by the cipher. | |
| 54 | 5. *MixColumns*: After the rows have been shifted, the columns are mixed and | |
| 55 | combined. | |
| 56 | ||
| 57 | This process does not necessarily stop after one full round. Steps 2 through 5 | |
| 58 | will repeat for the number of rounds specified by the key. However, the final | |
| 59 | round excludes the MixColumns step. As you can see, this is a fairly complex | |
| 60 | process. One must have a solid understanding of general mathematic principles to | |
| 61 | fully understand how the sequence works (and to even attempt to find a | |
| 62 | weakness). | |
| 63 | ||
| 64 | According to research done by Bogdanov et al., it would take billions of years | |
| 65 | to brute force a 126-bit key with current hardware. Additionally, this brute | |
| 66 | force attack would require storing 2^88 bits of data! However, there | |
| 67 | are a few different attacks that have been used to show vulnerabilities with the | |
| 68 | use of this technology. Side-channel attacks use inadvertent leaks of data from | |
| 69 | the hardware or software, which can allow attackers to obtain the key or run | |
| 70 | programs on a user's hardware. | |
| 71 | ||
| 72 | Please note that this is not something you should run out and try to implement | |
| 73 | in your `Hello, World!` app after only a few hours of research. While AES | |
| 74 | (basically all encryption methods) is extremely efficient in what it does, it | |
| 75 | takes a lot of time and patience to understand. If you're looking for something | |
| 76 | which currently implements AES, check out the [[https://www.bouncycastle.org/documentation.html][Legion of the Bouncy Castle]] for | |
| 77 | Java implementations of cryptographic algorithms. | |
| 78 | ||
| 79 | * Why Does Encryption Matter? | |
| 80 | ||
| 81 | There are limitless reasons to enable encryption at-rest or in-transit for | |
| 82 | various aspects of your digital life. You can research specific examples, such | |
| 83 | as [[https://arstechnica.com/tech-policy/2018/12/australia-passes-new-law-to-thwart-strong-encryption/][Australia passes new law to thwart strong encryption]]. However, I will simply | |
| 84 | list a few basic reasons to always enable encryption, where feasible: | |
| 85 | ||
| 86 | 1. Privacy is a human right and is recognized as a national right in some | |
| 87 | countries (e.g., [[https://www.law.cornell.edu/wex/fourth_amendment][US Fourth Amendment]]). | |
| 88 | 2. "Why not?" Encryption rarely affects performance or speed, so there's usually | |
| 89 | not a reason to avoid it in the first place. | |
| 90 | 3. Your digital identity and activity (texts, emails, phone calls, online | |
| 91 | accounts, etc.) are extremely valuable and can result in terrible | |
| 92 | consequences, such as identity theft, if leaked to other parties. Encrypting | |
| 93 | this data prevents such leaks from ruining lives. | |
| 94 | 4. Wiping or factory-resetting does not actually wipe all data from the storage | |
| 95 | device. There are methods to read data from the physical disks/boards inside | |
| 96 | devices. | |
| 97 | 5. Corporations, governments, and other nefarious groups/individuals are | |
| 98 | actively looking for ways to collect personal information about anyone they | |
| 99 | can. If someone's data is unencrypted, that person may become a target due to | |
| 100 | the ease of data collection. | |
| 101 | ||
| 102 | **Read More:** | |
| 103 | - [[http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.197.pdf][Federal Information Processing Standards Publication 197]] | |
blog/2019-01-07-useful-css.org added +181
| @@ -0,0 +1,181 @@ | ||
| 1 | #+date: 2019-01-07 | |
| 2 | #+title: Useful CSS Snippets | |
| 3 | ||
| 4 | * Introduction to CSS | |
| 5 | ||
| 6 | [[https://en.wikipedia.org/wiki/CSS][CSS]], the language used to markup HTML code and make it "pretty", is one of the | |
| 7 | most effective ways to increase the attractiveness of a website. It can also | |
| 8 | lead to increased user engagement, retention, and satisfaction. In fact, there | |
| 9 | are whole career fields are dedicated to the improvement of user experiences, | |
| 10 | known as UI design and UX design. | |
| 11 | ||
| 12 | Some web developers are used to the common CSS properties, such as element | |
| 13 | sizing, fonts, colors, etc., but are not as well versed in less-used properties | |
| 14 | and values such as =flexbox=, =clip-path=, and =transform=. This article will | |
| 15 | provide some insight into the less-used and unique CSS properties. | |
| 16 | ||
| 17 | * CSS Variables | |
| 18 | ||
| 19 | The first topic today is CSS variables. Variables are not often used by smaller | |
| 20 | developers. CSS variables allow you to give your website a well-defined | |
| 21 | structure, where you can easily reuse CSS properties throughout the project. | |
| 22 | ||
| 23 | You can use variables to define things, such as color palettes. Then, you can | |
| 24 | use these colors for backgrounds anywhere else in the HTML. This could be | |
| 25 | extended, where extra variables could be defined for =primary-text=, | |
| 26 | =quoted-text=, etc. Variables can also be used to define spacing (e.g. =32px= or | |
| 27 | =2rem=), which can then be applied to margins, padding, font sizes, and more. | |
| 28 | ||
| 29 | For example, here are some variables defined at the root of the website, which | |
| 30 | allows for any subsequent CSS rules to use those variables: | |
| 31 | ||
| 32 | #+BEGIN_SRC css | |
| 33 | :root { | |
| 34 | --primary-color: black; | |
| 35 | --secondary-color: white; | |
| 36 | } | |
| 37 | ||
| 38 | body { | |
| 39 | background-color: var(--primary-color); | |
| 40 | color: var(--secondary-color); | |
| 41 | } | |
| 42 | #+END_SRC | |
| 43 | ||
| 44 | * CSS Box Shadows | |
| 45 | ||
| 46 | Box shadows were once my mortal enemy. No matter how hard I tried, I just | |
| 47 | couldn't get them to work how I wanted. Because of this, my favorite discovery | |
| 48 | has been CSSMatic's [[https://www.cssmatic.com/box-shadow][box shadow generator]]. It provides an excellent tool to | |
| 49 | generate box shadows using their simple sliders. Surprisingly, this is the | |
| 50 | reason I learned how box shadows work! You can use the sliders and watch how the | |
| 51 | CSS code changes in the image that is displayed. Through this, you should | |
| 52 | understand that the basic structure for box shadows is: | |
| 53 | ||
| 54 | #+BEGIN_SRC css | |
| 55 | box-shadow: inset horizontal vertical blur spread color; | |
| 56 | #+END_SRC | |
| 57 | ||
| 58 | Now, let's look at some basic examples! You can copy and paste the following | |
| 59 | code into a site like CodePen or your own HTML files. Feel free to play around | |
| 60 | with the code, experiment, and learn. | |
| 61 | ||
| 62 | **Box Shadow #1** | |
| 63 | ||
| 64 | #+BEGIN_SRC html | |
| 65 | <div class="shadow-examples"> | |
| 66 | <div class="box effect1"> | |
| 67 | <h3>Effect 1</h3> | |
| 68 | </div> | |
| 69 | </div> | |
| 70 | #+END_SRC | |
| 71 | ||
| 72 | #+BEGIN_SRC css | |
| 73 | .box h3 { | |
| 74 | text-align: center; | |
| 75 | position: relative; | |
| 76 | top: 80px; | |
| 77 | } | |
| 78 | .box { | |
| 79 | width: 70%; | |
| 80 | height: 200px; | |
| 81 | background: #fff; | |
| 82 | margin: 40px auto; | |
| 83 | } | |
| 84 | .effect1 { | |
| 85 | box-shadow: 0 10px 6px -6px #777; | |
| 86 | } | |
| 87 | #+END_SRC | |
| 88 | ||
| 89 | **Box Shadow #2** | |
| 90 | ||
| 91 | #+BEGIN_SRC html | |
| 92 | <div class="shadow-examples"> | |
| 93 | <div class="box effect2"> | |
| 94 | <h3>Effect 2</h3> | |
| 95 | </div> | |
| 96 | </div> | |
| 97 | #+END_SRC | |
| 98 | ||
| 99 | #+BEGIN_SRC css | |
| 100 | .box h3 { | |
| 101 | text-align: center; | |
| 102 | position: relative; | |
| 103 | top: 80px; | |
| 104 | } | |
| 105 | .box { | |
| 106 | width: 70%; | |
| 107 | height: 200px; | |
| 108 | background: #fff; | |
| 109 | margin: 40px auto; | |
| 110 | } | |
| 111 | .effect2 { | |
| 112 | box-shadow: 10px 10px 5px -5px rgba(0, 0, 0, 0.75); | |
| 113 | } | |
| 114 | #+END_SRC | |
| 115 | ||
| 116 | Try these box shadows out on your own and see how changing each shadow value | |
| 117 | works. | |
| 118 | ||
| 119 | * CSS Flexbox | |
| 120 | ||
| 121 | Now, let's move on to the best part of this article: flexbox. The flexbox is by | |
| 122 | far my favorite new toy. I originally stumbled across this solution after | |
| 123 | looking for more efficient ways of centering content horizontally AND | |
| 124 | vertically. I had used a few hack-ish methods before, but flexbox throws those | |
| 125 | out the window. The best part of it all is that flexbox is /dead simple/. | |
| 126 | ||
| 127 | Flexbox pertains to the parent div of any element. You want the parent to be the | |
| 128 | flexbox in which items are arranged to use the flex methods. It's easier to see | |
| 129 | this in action that explained, so let's see an example. | |
| 130 | ||
| 131 | **Flexbox** | |
| 132 | ||
| 133 | #+BEGIN_SRC html | |
| 134 | <div class="flex-examples"> | |
| 135 | <div class="sm-box"> | |
| 136 | <h3>1</h3> | |
| 137 | </div> | |
| 138 | <div class="sm-box"> | |
| 139 | <h3>2</h3> | |
| 140 | </div> | |
| 141 | </div> | |
| 142 | #+END_SRC | |
| 143 | ||
| 144 | #+BEGIN_SRC css | |
| 145 | .flex-examples { | |
| 146 | display: flex; | |
| 147 | flex-wrap: wrap; | |
| 148 | justify-content: flex-start; | |
| 149 | align-items: center; | |
| 150 | padding: 10px; | |
| 151 | background-color: #f2f2f2; | |
| 152 | } | |
| 153 | .sm-box { | |
| 154 | display: flex; | |
| 155 | justify-content: center; | |
| 156 | align-items: center; | |
| 157 | width: 20%; | |
| 158 | height: 100px; | |
| 159 | background: #fff; | |
| 160 | margin: 40px 10px; | |
| 161 | } | |
| 162 | #+END_SRC | |
| 163 | ||
| 164 | You may notice that we no longer need to use the =top= property for the =h3= | |
| 165 | elements in our code. This is because we set the display box to be a flex | |
| 166 | container for the small boxes, AND we made the small boxes flex containers for | |
| 167 | their elements (the h3 tags). Flex boxes can be nested like this to center | |
| 168 | content that is inside centered content. | |
| 169 | ||
| 170 | For the example above, we designated the =justify-content= property to be | |
| 171 | =flex-start= so that the boxes stack from the left side of the screen. This | |
| 172 | property can be changed to =center= to make the boxes appear in the center of | |
| 173 | the screen. | |
| 174 | ||
| 175 | For an interactive example, [[https://codepen.io/LandonSchropp/pen/KpzzGo][check out this CodePen]] from [[https://codepen.io/LandonSchropp/][LandonScropp]]. Resize the | |
| 176 | window with dice to see how they collapse and re-align. | |
| 177 | ||
| 178 | * Even More CSS | |
| 179 | ||
| 180 | For more inspiration, you can visit [[https://www.codepen.io][CodePen]], [[https://dribbble.com][Dribbble]], or [[https://uimovement.com][UI Movement]] to browse | |
| 181 | the collections of many amazing web designers. | |
blog/2019-09-09-audit-analytics.org added +213
| @@ -0,0 +1,213 @@ | ||
| 1 | #+date: 2019-09-09 | |
| 2 | #+title: Data Analysis in Auditing | |
| 3 | ||
| 4 | * What Are Data Analytics? | |
| 5 | ||
| 6 | A quick aside before I dive into this post: `data analytics` is a vague term | |
| 7 | that has become popular in recent years. Think of a `data analytic` as the | |
| 8 | output of any data analysis you perform. For example, a pivot table or a pie | |
| 9 | chart could be a data analytic. | |
| 10 | ||
| 11 | [[https://en.wikipedia.org/wiki/Data_analysis][Data analysis]] is a process that utilizes statistics and other mathematical | |
| 12 | methods to discover useful information within datasets. This involves examining, | |
| 13 | cleaning, transforming, and modeling data so that you can use the data to | |
| 14 | support an opinion, create more useful viewpoints, and gain knowledge to | |
| 15 | implement into audit planning or risk assessments. | |
| 16 | ||
| 17 | One of the common mistakes that managers (and anyone new to the process) make is | |
| 18 | assuming that everything involved with this process is "data analytics". In | |
| 19 | fact, data analytics are only a small part of the process. | |
| 20 | ||
| 21 | See *Figure 1* for a more accurate representation of where data analysis sits | |
| 22 | within the full process. This means that data analysis does not include | |
| 23 | querying or extracting data, selecting samples, or performing audit tests. | |
| 24 | These steps can be necessary for an audit (and may even be performed by the same | |
| 25 | associates), but they are not data analytics. | |
| 26 | ||
| 27 | #+CAPTION: The Intelligence Cycle | |
| 28 | [[https://img.0x4b1d.org/blog/20190909-data-analysis-in-auditing/intelligence_cycle-min.png]] | |
| 29 | ||
| 30 | * Current Use of Analytics in Auditing | |
| 31 | ||
| 32 | While data analysis has been an integral part of most businesses and departments | |
| 33 | for the better part of the last century, only recently have internal audit | |
| 34 | functions been adopting this practice. The internal audit function works | |
| 35 | exclusively to provide assurance and consulting services to the business areas | |
| 36 | within the firm (except for internal auditing firms who are hired by different | |
| 37 | companies to perform their roles). | |
| 38 | ||
| 39 | #+BEGIN_QUOTE | |
| 40 | Internal Auditing helps an organization accomplish its objectives by bringing | |
| 41 | a systematic, disciplined approach to evaluate and improve the effectiveness | |
| 42 | of risk management, control and governance processes. | |
| 43 | ||
| 44 | - The IIA's Definition of Internal Audit | |
| 45 | #+END_QUOTE | |
| 46 | ||
| 47 | Part of the blame for the slow adoption of data analysis can be attributed to | |
| 48 | the fact that internal auditing is strongly based on tradition and following the | |
| 49 | precedents set by previous auditors. However, there can be no progress without | |
| 50 | auditors who are willing to break the mold and test new audit techniques. In | |
| 51 | fact, as of 2018, [[https://www.cpapracticeadvisor.com/accounting-audit/news/12404086/internal-audit-groups-are-lagging-in-data-analytics][only 63% of internal audit departments currently utilize data | |
| 52 | analytics]] in North America. This number should be as close as possible to | |
| 53 | 100%. I have never been part of an audit that would not have benefited from data | |
| 54 | analytics. | |
| 55 | ||
| 56 | So, how do internal audit functions remedy this situation? It's definitely not | |
| 57 | as easy as walking into work on Monday and telling your Chief Audit Executive | |
| 58 | that you're going to start implementing analytics in the next audit. You need a | |
| 59 | plan and a system to make the analysis process as effective as possible. | |
| 60 | ||
| 61 | * The DELTA Model | |
| 62 | ||
| 63 | One of the easiest ways to experiment with data analytics and gain an | |
| 64 | understanding of the processes is to implement them within your own department. | |
| 65 | But how do we do this if we've never worked with analysis before? One of the | |
| 66 | most common places to start is to research some data analysis models currently | |
| 67 | available. For this post, we'll take a look at the DELTA model. You can take a | |
| 68 | look at **Figure 2** for a quick overview of the model. | |
| 69 | ||
| 70 | The DELTA model sets a few guidelines for areas wanting to implement data | |
| 71 | analytics so that the results can be as comprehensive as possible: | |
| 72 | ||
| 73 | - *Data*: Must be clean, accessible, and (usually) unique. | |
| 74 | - *Enterprise-Wide Focus*: Key data systems and analytical resources must be | |
| 75 | available for use (by the Internal Audit Function). | |
| 76 | - *Leaders*: Must promote a data analytics approach and show the value of | |
| 77 | analytical results. | |
| 78 | - *Targets*: Must be set for key areas and risks that the analytics can be | |
| 79 | compared against (KPIs). | |
| 80 | - *Analysts*: There must be auditors willing and able to perform data analytics | |
| 81 | or else the system cannot be sustained. | |
| 82 | ||
| 83 | #+CAPTION: The Delta Model | |
| 84 | [[https://img.0x4b1d.org/blog/20190909-data-analysis-in-auditing/delta-min.png]] | |
| 85 | ||
| 86 | * Finding the Proper KPIs | |
| 87 | ||
| 88 | Once the Internal Audit Function has decided that they want to start using data | |
| 89 | analytics internally and have ensured they're properly set up to do so, they | |
| 90 | need to figure out what they will be testing against. Key Performance Indicators | |
| 91 | (KPIs) are qualitative or quantitative factors that can be evaluated and | |
| 92 | assessed to determine if the department is performing well, usually compared to | |
| 93 | historical or industry benchmarks. Once KPIs have been agreed upon and set, | |
| 94 | auditors can use data analytics to assess and report on these KPIs. This allows | |
| 95 | the person performing the analytics the freedom to express opinions on the | |
| 96 | results, whereas the results are ambiguous if no KPIs exist. | |
| 97 | ||
| 98 | It should be noted that tracking KPIs in the department can help ensure you have | |
| 99 | a rigorous Quality Assurance and Improvement Program (QAIP) in accordance with | |
| 100 | some applicable standards, such as IPPF Standard 1300. | |
| 101 | ||
| 102 | #+BEING_QUOTE | |
| 103 | The chief audit executive must develop and maintain a quality assurance and | |
| 104 | improvement program that covers all aspects of the internal audit activity. | |
| 105 | ||
| 106 | - IPPF Standard 1300 | |
| 107 | #+END_QUOTE | |
| 108 | ||
| 109 | Additionally, IPPF Standard 2060 discusses reporting: | |
| 110 | ||
| 111 | #+BEGIN_QUOTE | |
| 112 | The chief audit executive must report periodically to senior management and the | |
| 113 | board on the internal audit activity's purpose, authority, responsibility, and | |
| 114 | performance relative to its plan and on its conformance with the Code of Ethics | |
| 115 | and the Standards. Reporting must also include significant risk and control | |
| 116 | issues, including fraud risks, governance issues, and other matters that require | |
| 117 | the attention of senior management and/or the board. | |
| 118 | ||
| 119 | - IPPF Standard 2060 | |
| 120 | #+END_QUOTE | |
| 121 | ||
| 122 | The hardest part of finding KPIs is to determine which KPIs are appropriate for | |
| 123 | your department. Since every department is different and has different goals, | |
| 124 | KPIs will vary drastically between companies. To give you an idea of where to | |
| 125 | look, here are some ideas I came up with when discussing the topic with a few | |
| 126 | colleagues. | |
| 127 | ||
| 128 | - Efficiency/Budgeting: | |
| 129 | - Audit hours to staff utilization ratio (annual hours divided by total annual | |
| 130 | work hours). | |
| 131 | - Audit hours compared to the number of audits completed. | |
| 132 | - Time between audit steps or to complete the whole audit. E.g., time from | |
| 133 | fieldwork completion to audit report issuance. | |
| 134 | - Reputation: | |
| 135 | - The frequency that management has requested the services of the IAF. | |
| 136 | - Management, audit committee, or external audit satisfaction survey results. | |
| 137 | - Education, experience, certifications, tenure, and training of the auditors | |
| 138 | on staff. | |
| 139 | - Quality: | |
| 140 | - Number and frequency of audit findings. Assign monetary or numerical values, | |
| 141 | if possible. | |
| 142 | - Percentage of recommendations issued and implemented. | |
| 143 | - Planning: | |
| 144 | - Percentage or number of key risks audited per year or per audit. | |
| 145 | - Proportion of audit universe audited per year. | |
| 146 | ||
| 147 | * Data Analysis Tools | |
| 148 | ||
| 149 | Finally, to be able to analyze and report on the data analysis, auditors need to | |
| 150 | evaluate the tools at their disposal. There are many options available, but a | |
| 151 | few of the most common ones can easily get the job done. For example, almost | |
| 152 | every auditor already has access to Microsoft Excel. Excel is more powerful than | |
| 153 | most people give it credit for and can accomplish a lot of basic statistics | |
| 154 | without much work. If you don't know a lot about statistics but still want to | |
| 155 | see some of the more basic results, Excel is a great option. | |
| 156 | ||
| 157 | To perform more in-depth statistical analysis or to explore large datasets that | |
| 158 | Excel cannot handle, auditors will need to explore other options. The big three | |
| 159 | that have had a lot of success in recent years are Python, R, and ACL. ACL can | |
| 160 | be used as either a graphical tool (point and click) or as a scripting tool, | |
| 161 | where the auditor must write the scripts manually. Python and the R-language are | |
| 162 | solely scripting languages. | |
| 163 | ||
| 164 | The general trend in the data analytics environment is that if the tool allows | |
| 165 | you to do everything by clicking buttons or dragging elements, you won't be able | |
| 166 | to fully utilize the analytics you need. The most robust solutions are created | |
| 167 | by those who understand how to write the scripts manually. It should be noted | |
| 168 | that as the utility of a tool increases, it usually means that the learning | |
| 169 | curve for that tool will also be higher. It will take auditors longer to learn | |
| 170 | how to utilize Python, R, or ACL versus learning how to utilize Excel. | |
| 171 | ||
| 172 | * Visualization | |
| 173 | ||
| 174 | Once an auditor has finally found the right data, KPIs, and tools, they must | |
| 175 | report these results so that actions can be taken. Performing in-depth data | |
| 176 | analysis is only useful if the results are understood by the audiences of the | |
| 177 | data. The best way to create this understanding is to visualize the results of | |
| 178 | the data. Let's take a look at some of the best options to visualize and report | |
| 179 | the results you've found. | |
| 180 | ||
| 181 | Some of the most popular commercial tools for visualization are Microsoft | |
| 182 | PowerBI and Tableau Desktop. However, other tools exist such as JMP, Plotly, | |
| 183 | Qlikview, Alteryx, or D3. Some require commercial licenses while others are | |
| 184 | simply free to use. For corporate data, you may want to make sure that the tool | |
| 185 | does not communicate any of the data outside the company (such as cloud | |
| 186 | storage). I won't be going into depth on any of these tools since visualization | |
| 187 | is largely a subjective and creative experience, but remember to constantly | |
| 188 | explore new options as you repeat the process. | |
| 189 | ||
| 190 | Lastly, let's take a look at an example of data visualization. This example | |
| 191 | comes from a [[https://talent.works/2018/03/28/the-science-of-the-job-search-part-iii-61-of-entry-level-jobs-require-3-years-of-experience/][blog post written by Kushal Chakrabarti]] in 2018 about the percent | |
| 192 | of entry-level US jobs that require experience. *Figure 3* shows us an | |
| 193 | easy-to-digest picture of the data. We can quickly tell that only about 12.5% of | |
| 194 | entry-level jobs don't require experience. | |
| 195 | ||
| 196 | This is the kind of result that easily describes the data for you. However, make | |
| 197 | sure to include an explanation of what the results mean. Don't let the reader | |
| 198 | assume what the data means, especially if it relates to a complex subject. /Tell | |
| 199 | a story/ about the data and why the results matter. For example, *Figure 4* | |
| 200 | shows a part of the explanation the author gives to illustrate his point. | |
| 201 | ||
| 202 | #+CAPTION: Entry-Level Visualization | |
| 203 | [[https://img.0x4b1d.org/blog/20190909-data-analysis-in-auditing/vis_example-min.png]] | |
| 204 | ||
| 205 | #+CAPTION: Visualization Explanation | |
| 206 | [[https://img.0x4b1d.org/blog/20190909-data-analysis-in-auditing/vis_example_explanation-min.png]] | |
| 207 | ||
| 208 | * Wrap-Up | |
| 209 | ||
| 210 | While this is not an all-encompassing program that you can just adopt into your | |
| 211 | department, it should be enough to get anyone started on the process of | |
| 212 | understanding and implementing data analytics. Always remember to continue | |
| 213 | learning and exploring new options as your processes grow and evolve. | |
blog/2019-12-03-the-ansoff-matrix.org added +135
| @@ -0,0 +1,135 @@ | ||
| 1 | #+date: 2019-12-03 | |
| 2 | #+title: The Ansoff Matrix | |
| 3 | ||
| 4 | * Overview | |
| 5 | ||
| 6 | As the world of business evolves, managers must approach business planning and | |
| 7 | strategy with a contemporary mindset. According to Dess, McNamara, Eisner, and | |
| 8 | Lee, managers must be willing to adapt to the modern business environment by | |
| 9 | going beyond "'incremental management', whereby they view their job as making a | |
| 10 | series of small, minor changes to improve the efficiency of the firm's | |
| 11 | operations"[fn:1]. | |
| 12 | ||
| 13 | One reason that strategic management is crucial is because most businesses that | |
| 14 | fail in the United States each year fail due to a lack of strategic focus or | |
| 15 | direction[fn:2]. The rate of failure for businesses with poor strategies shows | |
| 16 | that strategic planning and management are crucial to a business's strength and | |
| 17 | longevity, injecting the critical factors of growth and direction into a | |
| 18 | company's business plan. | |
| 19 | ||
| 20 | One of the most significant strategic planning and management frameworks that | |
| 21 | companies can use is the [[https://en.wikipedia.org/wiki/Ansoff_matrix][Ansoff Matrix]]. While this framework has unique purposes | |
| 22 | and use-cases, it can effectively help an organization grow and | |
| 23 | compete. Specifically, the Ansoff matrix is one of the most effective frameworks | |
| 24 | for companies who want to focus on increasing sales revenue or | |
| 25 | profitability[fn:3]. | |
| 26 | ||
| 27 | This framework uses a two-by-two figure to show the four strategic options for | |
| 28 | companies to use in this framework: market penetration, market development, | |
| 29 | product development, and diversification (see *Figure 1*). The x-axis of the | |
| 30 | matrix focuses on the firm's markets and also determines if the firm is looking | |
| 31 | to enter new markets or innovate in its current markets. The y-axis of the | |
| 32 | matrix focuses on the firm's products and determines if the firm wants to pursue | |
| 33 | strategies around their existing products or explore new products. | |
| 34 | ||
| 35 | #+CAPTION: The Ansoff Matrix by JaisonAbeySabu, Own work, CC BY-SA 3.0 | |
| 36 | [[https://img.0x4b1d.org/blog/20191203-the-ansoff-matrix/ansoff_matrix-min.png]] | |
| 37 | ||
| 38 | * Strategic Options | |
| 39 | ||
| 40 | ** Market Penetration | |
| 41 | ||
| 42 | The most straightforward strategy in the Ansoff matrix is to focus on existing | |
| 43 | products in existing markets, also known as market penetration[fn:3]. Companies | |
| 44 | such as Coca-Cola have used market penetration successfully by investing a lot | |
| 45 | of money to get further value out of their current markets. Coca-Cola does this | |
| 46 | by introducing new features such as Christmas-themed bottles, personal names on | |
| 47 | the bottles, and other marketing schemes. | |
| 48 | ||
| 49 | ** Market Development | |
| 50 | ||
| 51 | Market development extends existing products into new markets in an attempt to | |
| 52 | increase the number of buyers. One interesting way that Coca-Cola used this | |
| 53 | strategy comes from the stigma that Diet Coke is a woman's drink[fn:4]. Coca-Cola | |
| 54 | introduced Coca-Cola Zero, which contained the same nutritional content as Diet | |
| 55 | Coke, but was packaged in a dark black can to appear more "manly"[fn:4]. | |
| 56 | ||
| 57 | ** Product Development | |
| 58 | ||
| 59 | Product development uses existing markets to introduce new products so that the | |
| 60 | firm can better meet customer needs[fn:4]. The extreme end of diversification is | |
| 61 | home to companies such as Johnson & Johnson, a healthcare company that has | |
| 62 | developed a business portfolio of more than 60,000 different products[fn:5]. | |
| 63 | Johnson & Johnson's dedication to continuous diversification has led them to a | |
| 64 | balance sheet rating of "AAA", industry recognition for diversification, and | |
| 65 | increases in their investor dividends for 57 consecutive years[fn:6]. | |
| 66 | ||
| 67 | ** Related Diversification | |
| 68 | ||
| 69 | Diversification, the final strategy of the Ansoff Matrix, is more difficult than | |
| 70 | the others since it involves exploring both new markets and new products. | |
| 71 | Related diversification is a diversification strategy that closely relates to | |
| 72 | the firm's core business. Coca-Cola's best example of related diversification is | |
| 73 | its acquisition of Glaceau and Vitamin Water, which expanded their drinking | |
| 74 | lines of business[fn:4]. | |
| 75 | ||
| 76 | ** Unrelated Diversification | |
| 77 | ||
| 78 | Unrelated diversification is a diversification strategy that does not really | |
| 79 | relate to the firm's core business but still diversifies their business | |
| 80 | portfolio. A good example of this would be a coffee company who has decided to | |
| 81 | enter the market for bicycle sales. The main purpose of this strategy is to an | |
| 82 | extremely diverse company that will not go bankrupt if one market goes through | |
| 83 | difficult times. However, this requires a lot of independent skills and heavy | |
| 84 | investments since the company most likely cannot easily transfer knowledge | |
| 85 | between the markets they compete in. | |
| 86 | ||
| 87 | * Requirements for Success | |
| 88 | ||
| 89 | To use the Ansoff Matrix framework, managers need to formulate corporate goals | |
| 90 | and objectives. Without goals and direction, management frameworks do not | |
| 91 | present much practical utility. Further, the Ansoff Matrix requires the managers | |
| 92 | involved to make tactical decisions and create a path for the company to take | |
| 93 | toward their goals. Lastly, both the Ansoff Matrix needs to consider both | |
| 94 | internal and external perspectives throughout the strategy formulation process. | |
| 95 | ||
| 96 | One interesting probability is that companies will be using multiple strategic | |
| 97 | planning and management frameworks at the same time. While this may sound like | |
| 98 | it could crowd the management process, there are numerous reasons to do so. For | |
| 99 | example, the Ansoff Matrix and the Balanced Scorecard are relatively popular, | |
| 100 | and they cover entirely different parts of a company's strategy. Using the | |
| 101 | results from the Balanced Scorecard could inform a company of the potential | |
| 102 | product and market demands, such as from customer or supplier survey results, | |
| 103 | to help the company determine which Ansoff Matrix strategy to pursue. | |
| 104 | However, a combined approach at this level would require mature frameworks | |
| 105 | and focused managers who are able to strategize at a high level. | |
| 106 | ||
| 107 | Lastly, it should be noted that the author of the Ansoff matrix, Igor Ansoff, | |
| 108 | often used the term [[https://en.wikipedia.org/wiki/Analysis_paralysis][paralysis by analysis]] to explain the mistake of companies | |
| 109 | who overuse analysis and spend too much time planning. Companies need to | |
| 110 | understand the utility of a strategic management framework while ensuring that | |
| 111 | the company is poised to execute as efficiently as they have planned. | |
| 112 | ||
| 113 | * Footnotes | |
| 114 | ||
| 115 | [fn:1]: Dess, G. G., McNamara, G., Eisner, A. B., Lee, S. H. (2019). Strategic | |
| 116 | management: Text & cases, ninth edition. New York, NY: McGraw-Hill Education. | |
| 117 | ||
| 118 | [fn:2]: Juneja, P. (n.d.). Benefits of strategic management. Management Study | |
| 119 | Guide. Retrieved from | |
| 120 | https://www.managementstudyguide.com/strategic-management-benefits.htm. | |
| 121 | ||
| 122 | [fn:3]: Meldrum M., McDonald M. (1995) The Ansoff matrix. In: Key Marketing | |
| 123 | Concepts. London: Palgrave. | |
| 124 | ||
| 125 | [fn:4]: Oakley, T. (2015). Coca-Cola: The Ansoff matrix. The Marketing Agenda. | |
| 126 | Retrieved from | |
| 127 | https://themarketingagenda.com/2015/03/28/coca-cola-ansoff-matrix/. | |
| 128 | ||
| 129 | [fn:5]: Lemke, T. (2019). The most diversified companies in the stock market. The | |
| 130 | balance. Retrieved from | |
| 131 | https://www.thebalance.com/the-most-diversified-companies-in-the-stock-market-4169730. | |
| 132 | ||
| 133 | [fn:6]: Johnson & Johnson. (2018). 2018 Investor Fact Sheet. [PDF file]. Retrieved | |
| 134 | from | |
| 135 | http://www.investor.jnj.com/\_document/2018-investor-fact-sheet-4-19'id=0000016a-5681-d475-a17f-d78db54a0000. | |
blog/2019-12-16-password-security.org added +107
| @@ -0,0 +1,107 @@ | ||
| 1 | #+date: 2019-12-16 | |
| 2 | #+title: Password Security | |
| 3 | ||
| 4 | * Users | |
| 5 | ||
| 6 | ** Why Does It Matter? | |
| 7 | ||
| 8 | Information security, including passwords and identities, has become one of the | |
| 9 | most important digital highlights of the last decade. With [[https://www.usatoday.com/story/money/2018/12/28/data-breaches-2018-billions-hit-growing-number-cyberattacks/2413411002/][billions of people | |
| 10 | affected by data breaches each year]], there's a greater need to introduce strong | |
| 11 | information security systems. If you think you've been part of a breach, or you | |
| 12 | want to check and see, you can use [[https://haveibeenpwned.com/][Have I Been Pwned]] to see if your email has | |
| 13 | been involved in any public breaches. Remember that there's a possibility that a | |
| 14 | company experienced a breach and did not report it to anyone. | |
| 15 | ||
| 16 | ** How Do I Protect Myself? | |
| 17 | ||
| 18 | The first place to start with any personal security check-up is to gather a list | |
| 19 | of all the different websites, apps, or programs that require you to have login | |
| 20 | credentials. Optionally, once you know where your information is being stored, | |
| 21 | you can sort the list from the most-important items such as banks or government | |
| 22 | logins to less important items such as your favorite meme site. You will want to | |
| 23 | ensure that your critical logins are secure before getting to the others. | |
| 24 | ||
| 25 | Once you think you have a good idea of all your different authentication | |
| 26 | methods, I recommend using a password manager such as [[https://bitwarden.com/][Bitwarden]]. Using a | |
| 27 | password manager allows you to automatically save your logins, create randomized | |
| 28 | passwords, and transfer passwords across devices. However, you'll need to | |
| 29 | memorize your "vault password" that allows you to open the password manager. | |
| 30 | It's important to make this something hard to guess since it would allow anyone | |
| 31 | who has it to access every password you've stored in there. | |
| 32 | ||
| 33 | Personally, I recommend using a [[https://en.wikipedia.org/wiki/Passphrase][passphrase]] instead of a [[https://en.wikipedia.org/wiki/Password][password]] for your vault | |
| 34 | password. Instead of using a string of characters (whether random or simple), | |
| 35 | use a phrase and add in symbols and a number. For example, your vault password | |
| 36 | could be =Racing-Alphabet-Gourd-Parrot3=. Swap the symbols out for whichever | |
| 37 | symbol you want, move the number around, and fine-tune the passphrase until you | |
| 38 | are confident that you can remember it whenever necessary. | |
| 39 | ||
| 40 | Once you've stored your passwords, make sure you continually check up on your | |
| 41 | account and make sure you aren't following bad password practices. Krebs on | |
| 42 | Security has a great [[https://krebsonsecurity.com/password-dos-and-donts/][blog post on password recommendations]]. Any time that a data | |
| 43 | breach happens, make sure you check to see if you were included, and if you need | |
| 44 | to reset any account passwords. | |
| 45 | ||
| 46 | * Developers | |
| 47 | ||
| 48 | ** What Are the Basic Requirements? | |
| 49 | ||
| 50 | When developing any password-protected application, there are a few basic rules | |
| 51 | that anyone should follow even if they do not follow any official guidelines | |
| 52 | such as NIST. The foremost practice is to require users to use passwords that | |
| 53 | are at least 8 characters and cannot easily be guessed. This sounds extremely | |
| 54 | simple, but it requires quite a few different strategies. First, the application | |
| 55 | should check the potential passwords against a dictionary of insecure passwords | |
| 56 | such =password=, =1234abc=, or =application_name=. | |
| 57 | ||
| 58 | Next, the application should offer guidance on the strength of passwords being | |
| 59 | entered during enrollment. Further, NIST officially recommends *not* | |
| 60 | implementing any composition rules that make passwords hard to remember (e.g. | |
| 61 | passwords with letters, numbers, and special characters) and instead encouraging | |
| 62 | the use of long pass phrases which can include spaces. It should be noted that | |
| 63 | to be able to keep spaces within passwords, all unicode characters should be | |
| 64 | supported, and passwords should not be truncated. | |
| 65 | ||
| 66 | ** What Does NIST Recommend? | |
| 67 | ||
| 68 | The National Institute of Standards and Technology ([[https://www.nist.gov][NIST]]) in the US Department | |
| 69 | of Commerce regularly publishes information around information security and | |
| 70 | digital identity guidelines. Recently, NIST published [[https://pages.nist.gov/800-63-3/sp800-63b.html][Special Publication | |
| 71 | 800-63b]]: Digital Identity Guidelines and Authentication and Lifecycle | |
| 72 | Management. | |
| 73 | ||
| 74 | #+BEGIN_QUOTE | |
| 75 | A Memorized Secret authenticator - commonly referred to as a password or, if | |
| 76 | numeric, a PIN - is a secret value intended to be chosen and memorized by the | |
| 77 | user. Memorized secrets need to be of sufficient complexity and secrecy that | |
| 78 | it would be impractical for an attacker to guess or otherwise discover the | |
| 79 | correct secret value. A memorized secret is something you know. | |
| 80 | ||
| 81 | - NIST Special Publication 800-63B | |
| 82 | #+END_QUOTE | |
| 83 | ||
| 84 | NIST offers a lot of guidance on passwords, but I'm going to highlight just a | |
| 85 | few of the important factors: | |
| 86 | ||
| 87 | - Require passwords to be a minimum of 8 characters (6 characters if randomly | |
| 88 | generated and be generated using an approved random bit generator). | |
| 89 | - Compare potential passwords against a list that contains values known to be | |
| 90 | commonly-used, expected, or compromised. | |
| 91 | - Offer guidance on password strength, such as a strength meter. | |
| 92 | - Implement a rate-limiting mechanism to limit the number of failed | |
| 93 | authentication attempts for each user account. | |
| 94 | - Do not require composition rules for passwords and do not require passwords to | |
| 95 | be changed periodically (unless compromised). | |
| 96 | - Allow pasting of user identification and passwords to facilitate the use of | |
| 97 | password managers. | |
| 98 | - Allow users to view the password as it is being entered. | |
| 99 | - Use secure forms of communication and storage, including salting and hashing | |
| 100 | passwords using a one-way key derivation function. | |
| 101 | ||
| 102 | NIST offers further guidance on other devices that require specific security | |
| 103 | policies, querying for passwords, and more. All the information discussed so far | |
| 104 | comes from [[https://pages.nist.gov/800-63-3/sp800-63b.html][NIST SP800-63b]] but NIST offers a lot of information on digital | |
| 105 | identities, enrollment, identity proofing, authentication, lifecycle management, | |
| 106 | federation, and assertions in the total [[https://pages.nist.gov/800-63-3/][NIST SP800-63 Digital Identity | |
| 107 | Guidelines]]. | |
blog/2020-01-25-linux-software.org added +250
| @@ -0,0 +1,250 @@ | ||
| 1 | #+date: 2020-01-25 | |
| 2 | #+title: Linux Software | |
| 3 | ||
| 4 | * GUI Applications | |
| 5 | ||
| 6 | ** Etcher | |
| 7 | ||
| 8 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/etcher.png]] | |
| 9 | ||
| 10 | [[https://www.balena.io/etcher/][Etcher]] is a quick and easy way to burn ISO images to CDs and USB devices. There | |
| 11 | are two different ways you can install this program. First, you can navigate to | |
| 12 | the [[https://www.balena.io/etcher/][official website]] and download the AppImage file, which can run without | |
| 13 | installation. | |
| 14 | ||
| 15 | However, AppImage files are not executable by default, so you'll either need to | |
| 16 | right-click to open the properties of the file and click the "Allow executing | |
| 17 | file as program" box in the Permissions tab or use the following command: | |
| 18 | ||
| 19 | #+BEGIN_SRC sh | |
| 20 | chmod u+x FILE_NAME | |
| 21 | #+END_SRC | |
| 22 | ||
| 23 | If you don't like AppImage files or just prefer repositories, you can use the | |
| 24 | following commands to add the author's repository and install it through the | |
| 25 | command-line only. | |
| 26 | ||
| 27 | First, you'll have to echo the repo and write it to a list file: | |
| 28 | ||
| 29 | #+BEGIN_SRC sh | |
| 30 | echo "deb https://deb.etcher.io stable etcher" | sudo tee /etc/apt/sources.list.d/balena-etcher.list | |
| 31 | #+END_SRC | |
| 32 | ||
| 33 | Next, add the application keys to Ubuntu's keyring: | |
| 34 | ||
| 35 | #+BEGIN_SRC sh | |
| 36 | sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 379CE192D401AB61 | |
| 37 | #+END_SRC | |
| 38 | ||
| 39 | Finally, update the repositories and install the app. | |
| 40 | ||
| 41 | #+BEGIN_SRC sh | |
| 42 | sudo apt update && sudo apt install balena-etcher-electron | |
| 43 | #+END_SRC | |
| 44 | ||
| 45 | Using Arch, Manjaro, or another distro using the AUR? Use this command instead: | |
| 46 | ||
| 47 | #+BEGIN_SRC sh | |
| 48 | sudo pacman -S etcher | |
| 49 | #+END_SRC | |
| 50 | ||
| 51 | ** Atom | |
| 52 | ||
| 53 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/atom.png]] | |
| 54 | ||
| 55 | [[https://atom.io][Atom]] is the self-proclaimed "hackable text editor for the 21st century". This | |
| 56 | text editor is made by GitHub, [[https://news.microsoft.com/2018/06/04/microsoft-to-acquire-github-for-7-5-billion/][now owned by Microsoft]], and has some of the best | |
| 57 | add-ons available to customize the layout and abilities of the app. | |
| 58 | ||
| 59 | First, add the Atom repository to your sources. | |
| 60 | ||
| 61 | #+BEGIN_SRC sh | |
| 62 | sudo add-apt-repository ppa:webupd8team/atom | |
| 63 | #+END_SRC | |
| 64 | ||
| 65 | Next, update your package listings and install atom. | |
| 66 | ||
| 67 | #+BEGIN_SRC sh | |
| 68 | sudo apt update && sudo apt install atom | |
| 69 | #+END_SRC | |
| 70 | ||
| 71 | If you have issues updating your packages with the Atom repository, you'll need | |
| 72 | to use the snap package described below instead of the repository. To remove the | |
| 73 | repository we just added, use this command: | |
| 74 | ||
| 75 | #+BEGIN_SRC sh | |
| 76 | sudo add-apt-repository -r ppa:webupd8team/atom | |
| 77 | #+END_SRC | |
| 78 | ||
| 79 | You can also install Atom as a snap package, but it must be installed with the | |
| 80 | `--classic` flag. A [[https://language-bash.com/blog/how-to-snap-introducing-classic-confinement][full explanation is available]] if you'd like to read more | |
| 81 | about why you need the classic flag. | |
| 82 | ||
| 83 | #+BEGIN_SRC sh | |
| 84 | snap install atom --classic | |
| 85 | #+END_SRC | |
| 86 | ||
| 87 | Using Arch, Manjaro, or another distro using the AUR? Use this command instead: | |
| 88 | ||
| 89 | #+BEGIN_SRC sh | |
| 90 | sudo pacman -S atom | |
| 91 | #+END_SRC | |
| 92 | ||
| 93 | --- | |
| 94 | ||
| 95 | ** Visual Studio Code | |
| 96 | ||
| 97 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/vscode.png]] | |
| 98 | ||
| 99 | [[https://code.visualstudio.com][Visual Studio Code]] is yet another fantastic choice for programming on Linux, | |
| 100 | especially if you need those extra add-ons to spice up your late-night coding | |
| 101 | sessions. The theme used in the screenshot is [[https://marketplace.visualstudio.com/items?itemName=EliverLara.mars][Mars]] by theme creator [[https://github.com/EliverLara][Eliver Lara]], | |
| 102 | who makes a ton of great themes for VS Code, Atom, and various Linux desktop | |
| 103 | environments. | |
| 104 | ||
| 105 | To install VS Code, you'll need to download the `.deb` file from the official | |
| 106 | website. Once you've downloaded the file, either double-click it to install | |
| 107 | through the Software Center or run the following command: | |
| 108 | ||
| 109 | #+BEGIN_SRC sh | |
| 110 | sudo dpkg -i FILE_NAME.deb | |
| 111 | #+END_SRC | |
| 112 | ||
| 113 | You can also install VS Code as a snap package, but it must be installed with | |
| 114 | the `--classic` flag. A [[https://language-bash.com/blog/how-to-snap-introducing-classic-confinement][full explanation is available]] if you'd like to read more | |
| 115 | about why you need the classic flag. | |
| 116 | ||
| 117 | #+BEGIN_SRC sh | |
| 118 | snap install code --classic | |
| 119 | #+END_SRC | |
| 120 | ||
| 121 | Using Arch, Manjaro, or another distro using the AUR? Use these commands | |
| 122 | instead: | |
| 123 | ||
| 124 | #+BEGIN_SRC sh | |
| 125 | sudo pacman -S yay binutils make gcc pkg-config fakeroot yay -S visual-studio-code-bin | |
| 126 | #+END_SRC | |
| 127 | ||
| 128 | ** GNOME Tweaks | |
| 129 | ||
| 130 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/gnome-tweaks.png]] | |
| 131 | ||
| 132 | [[https://gitlab.gnome.org/GNOME/gnome-tweaks][Gnome Tweaks]] is the ultimate tool to use if you want to customize your GNOME | |
| 133 | desktop environment. This is how you can switch application themes (GTK), shell | |
| 134 | themes, icons, fonts, and more. To install GNOME Tweaks on Ubuntu, you just need | |
| 135 | to install the official package. | |
| 136 | ||
| 137 | #+BEGIN_SRC sh | |
| 138 | sudo apt install gnome-tweaks | |
| 139 | #+END_SRC | |
| 140 | ||
| 141 | If you've installed Manjaro or Arch with Gnome, you should have the tweak tool | |
| 142 | pre-installed. If you're on Fedora, this tool is available as an official | |
| 143 | package: | |
| 144 | ||
| 145 | #+BEGIN_SRC sh | |
| 146 | sudo dnf install gnome-tweaks | |
| 147 | #+END_SRC | |
| 148 | ||
| 149 | ** Steam | |
| 150 | ||
| 151 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/steam.png]] | |
| 152 | ||
| 153 | [[https://steampowered.com][Steam]] is one of the most popular gaming libraries for computers and is one of | |
| 154 | the main reasons that many people have been able to switch to Linux in recent | |
| 155 | years, thanks to Steam Proton, which makes it easier to play games not | |
| 156 | officially created for Linux platforms. | |
| 157 | ||
| 158 | To install Steam on Ubuntu, you just need to install the official package. | |
| 159 | ||
| 160 | #+BEGIN_SRC sh | |
| 161 | sudo apt install steam-installer | |
| 162 | #+END_SRC | |
| 163 | ||
| 164 | For Arch-based systems, you'll simply need to install the =steam= package. | |
| 165 | However, this requires that you enable the =multilib= source. To do so, use the | |
| 166 | following command: | |
| 167 | ||
| 168 | #+BEGIN_SRC sh | |
| 169 | sudo nano /etc/pacman.conf | |
| 170 | #+END_SRC | |
| 171 | ||
| 172 | Now, scroll down and uncomment the =multilib= section. | |
| 173 | ||
| 174 | #+BEGIN_SRC config | |
| 175 | # Before: | |
| 176 | #[multilib] | |
| 177 | #Include = /etc/pacman.d/mirrorlist | |
| 178 | ||
| 179 | # After: | |
| 180 | [multilib] | |
| 181 | Include = /etc/pacman.d/mirrorlist | |
| 182 | #+END_SRC | |
| 183 | ||
| 184 | Finally, install the program: | |
| 185 | ||
| 186 | #+BEGIN_SRC sh | |
| 187 | sudo pacman -S steam | |
| 188 | #+END_SRC | |
| 189 | ||
| 190 | [[./2020-01-26-steam-on-ntfs-drives.html][Problem Launching Steam Games? Click Here.]] | |
| 191 | ||
| 192 | * Command-Line Packages | |
| 193 | ||
| 194 | ** neofetch | |
| 195 | ||
| 196 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/neofetch.png]] | |
| 197 | ||
| 198 | [[https://github.com/dylanaraps/neofetch][Neofetch]] is a customizable tool used in the command-line to show system | |
| 199 | information. This is exceptionally useful if you want to see your system's | |
| 200 | information quickly without the clutter of some resource-heavy GUI apps. | |
| 201 | ||
| 202 | This is an official package if you're running Ubuntu 17.04 or later, so simply | |
| 203 | use the following command: | |
| 204 | ||
| 205 | #+BEGIN_SRC sh | |
| 206 | sudo apt install neofetch | |
| 207 | #+END_SRC | |
| 208 | ||
| 209 | If you're running Ubuntu 16.10 or earlier, you'll have to use a series of | |
| 210 | commands: | |
| 211 | ||
| 212 | #+BEGIN_SRC sh | |
| 213 | sudo add-apt-repository ppa:dawidd0811/neofetch; sudo apt update; sudo apt install neofetch | |
| 214 | #+END_SRC | |
| 215 | ||
| 216 | Using Arch, Manjaro, or another distro using the AUR? Use this command instead: | |
| 217 | ||
| 218 | #+BEGIN_SRC sh | |
| 219 | sudo pacman -S neofetch | |
| 220 | #+END_SRC | |
| 221 | ||
| 222 | ** yt-dlp | |
| 223 | ||
| 224 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/yt-dlp.png]] | |
| 225 | ||
| 226 | [[https://github.com/yt-dlp/yt-dlp][yt-dlp]] is an extremely handy command-line tool that allows you to download video | |
| 227 | or audio files from various websites, such as YouTube. There are a ton of | |
| 228 | different options when running this package, so be sure to run `yt-dlp --help` | |
| 229 | first to look through everything you can do (or give up and search for the best | |
| 230 | config online). | |
| 231 | ||
| 232 | While this shouldn't be a problem for most users, yt-dlp requires Python 2.6, | |
| 233 | 2.7, or 3.2+ to work correctly, so install Python if you don't have it already. | |
| 234 | You can check to see if you have Python installed by running: | |
| 235 | ||
| 236 | #+BEGIN_SRC sh | |
| 237 | python -V | |
| 238 | #+END_SRC | |
| 239 | ||
| 240 | To get the youtube-dl package, simply curl the URL and output the results. | |
| 241 | ||
| 242 | #+BEGIN_SRC sh | |
| 243 | sudo curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp | |
| 244 | #+END_SRC | |
| 245 | ||
| 246 | Finally, make the file executable so that it can be run from the command-line. | |
| 247 | ||
| 248 | #+BEGIN_SRC sh | |
| 249 | sudo chmod a+rx /usr/local/bin/yt-dlp | |
| 250 | #+END_SRC | |
blog/2020-01-26-steam-on-ntfs.org added +88
| @@ -0,0 +1,88 @@ | ||
| 1 | #+date: 2020-01-26 | |
| 2 | #+title: Linux Gaming Tweak: Steam on NTFS Drives | |
| 3 | ||
| 4 | * Auto-Mount Steam Drives | |
| 5 | ||
| 6 | [[https://img.0x4b1d.org/blog/20200125-the-best-linux-software/steam.png]] | |
| 7 | ||
| 8 | If you want to see how to install Steam on Linux, see my other post: [[./2020-01-25-linux-software.html][Linux Software]]. | |
| 9 | ||
| 10 | Are you having trouble launching games, even though they've installed correctly? | |
| 11 | This may happen if you're storing your games on an NTFS-formatted drive. This | |
| 12 | shouldn't be an issue if you're storing your games on the same drive that Steam | |
| 13 | is on, but some gamers prefer to put Steam on their main drive and game files on | |
| 14 | another SSD or HDD. | |
| 15 | ||
| 16 | To fix this problem, you'll need to try a few things. First, you'll need to | |
| 17 | install the =ntfs-3g= package, which is meant for better interoperability with | |
| 18 | Linux. | |
| 19 | ||
| 20 | #+BEGIN_SRC sh | |
| 21 | sudo apt install ntfs-3g | |
| 22 | #+END_SRC | |
| 23 | ||
| 24 | Next, you should set up the =/etc/fstab= file to automatically mount your drives | |
| 25 | on boot. To automatically mount your drives when the computer boots up, you'll | |
| 26 | have to create the folders you want to mount your drive to first. I store mine | |
| 27 | in the =/mnt= folder using names that I'll recognize, but you can create your | |
| 28 | folders wherever you want. | |
| 29 | ||
| 30 | #+BEGIN_SRC sh | |
| 31 | mkdir /path/to/folder | |
| 32 | #+END_SRC | |
| 33 | ||
| 34 | For example: | |
| 35 | ||
| 36 | #+BEGIN_SRC sh | |
| 37 | mkdir /mnt/steam_library | |
| 38 | #+END_SRC | |
| 39 | ||
| 40 | To automatically mount drives upon system boot, you will need to collect a few | |
| 41 | items. The UUID is the identification number connected to whichever drive you're | |
| 42 | using to store Steam games. | |
| 43 | ||
| 44 | Drives are usually labeled similar to =/dev/nvme0n1p1= or =/dev/sda1=, so you'll | |
| 45 | need to find the line in the output of the command below that correlates to your | |
| 46 | drive and copy the UUID over to the =/etc/fstab= file. | |
| 47 | ||
| 48 | #+BEGIN_SRC sh | |
| 49 | sudo blkid | grep UUID= | |
| 50 | #+END_SRC | |
| 51 | ||
| 52 | Next, you'll need your =uid= and =gid=. To find these, run the following | |
| 53 | command: | |
| 54 | ||
| 55 | #+BEGIN_SRC sh | |
| 56 | id -u && id -g | |
| 57 | #+END_SRC | |
| 58 | ||
| 59 | Now that you have collected the necessary information, open the `/etc/fstab` | |
| 60 | file: | |
| 61 | ||
| 62 | #+BEGIN_SRC sh | |
| 63 | sudo nano /etc/fstab | |
| 64 | #+END_SRC | |
| 65 | ||
| 66 | Each drive you want to mount on boot should have its own line in the | |
| 67 | =/etc/fstab= file that looks similar to this: | |
| 68 | ||
| 69 | #+BEGIN_SRC config | |
| 70 | UUID=B64E53824E5339F7 /mnt/steam_library ntfs-3g uid=1000,gid=1000 0 0 | |
| 71 | #+END_SRC | |
| 72 | ||
| 73 | Now all you need to do is unmount your drive and re-mount it. You can unmount | |
| 74 | the drive by doing this (be sure to use the correct drive name here): | |
| 75 | ||
| 76 | #+BEGIN_SRC sh | |
| 77 | sudo umount /dev/sdxX | |
| 78 | #+END_SRC | |
| 79 | ||
| 80 | You can re-mount all your drives by executing the following: | |
| 81 | ||
| 82 | #+BEGIN_SRC sh | |
| 83 | sudo mount -a | |
| 84 | #+END_SRC | |
| 85 | ||
| 86 | If you don't know what your drive name is, or you're nervous about unmounting | |
| 87 | and re-mounting, simply reboot your computer, and it will be done for you | |
| 88 | automatically. | |
blog/2020-02-09-cryptography-basics.org added +163
| @@ -0,0 +1,163 @@ | ||
| 1 | #+date: 2020-02-09 | |
| 2 | #+title: Cryptography Basics | |
| 3 | ||
| 4 | * Similar Article Available | |
| 5 | ||
| 6 | If you haven't already, feel free to read my post on [[./2018-12-08-aes-encryption.html][AES Encryption]]. | |
| 7 | ||
| 8 | * What is Cryptography? | |
| 9 | ||
| 10 | In layman's terms, cryptography is a process that can change data from a | |
| 11 | readable format into an unreadable format (and vice-versa) through a series of | |
| 12 | processes and secrets. More technically, this is the Internet Security | |
| 13 | Glossary's definition: | |
| 14 | ||
| 15 | #+BEGIN_QUOTE | |
| 16 | [Cryptography is] the mathematical science that deals with transforming data | |
| 17 | to render its meaning unintelligible (i.e., to hide its semantic content), | |
| 18 | prevent its undetected alteration, or prevent its unauthorized use. If the | |
| 19 | transformation is reversible, cryptography also deals with restoring encrypted | |
| 20 | data to an intelligible form. | |
| 21 | ||
| 22 | - [[https://tools.ietf.org/html/rfc2828][Internet Security Glossary (2000)]] | |
| 23 | #+END_QUOTE | |
| 24 | ||
| 25 | Cryptography cannot offer protection against the loss of data; it simply offers | |
| 26 | encryption methods to protect data at-rest and data in-traffic. At a high-level, | |
| 27 | encrypted is when plaintext data is encrypted to ciphertext (a secure form of | |
| 28 | text that cannot be understood unless decrypted back to plaintext). The | |
| 29 | encryption process is completed through the use of a mathematical function that | |
| 30 | utilizes one or more values called keys to encrypt or decrypt the data. | |
| 31 | ||
| 32 | * Key Elements of Cryptographic Systems | |
| 33 | ||
| 34 | To create or evaluate a cryptographic system, you need to know the essential | |
| 35 | pieces to the system: | |
| 36 | ||
| 37 | - *Encryption Algorithm (Primitive):* A mathematical process that encrypts and | |
| 38 | decrypts data. | |
| 39 | - *Encryption Key:* A string of bits used within the encryption algorithm as | |
| 40 | the secret that allows successful encryption or decryption of data. | |
| 41 | - *Key Length (Size):* The maximum number of bits within the encryption key. | |
| 42 | It's important to remember that key size is regulated in many countries. | |
| 43 | - *Message Digest:* A smaller, fixed-size bit string version of the original | |
| 44 | message. This is practically infeasible to reverse, which is why it's commonly | |
| 45 | used to verify integrity. | |
| 46 | ||
| 47 | * Symmetric Systems (Secret Key Cryptography) | |
| 48 | ||
| 49 | Symmetric cryptography utilizes a secret, bidirectional key to perform both | |
| 50 | encryption and decryption of the data. The most common implementation of | |
| 51 | symmetric cryptography is the Advanced Encryption Standard, which uses keys that | |
| 52 | are 128 bits to 256 bits in size. This standard came after the National | |
| 53 | Institute of Standards and Technology (NIST) decided to retire the Data | |
| 54 | Encryption Standard (DES) in 2001. | |
| 55 | ||
| 56 | Since brute force attacks strongly correlate with key length, the 56-bit key | |
| 57 | length of DES was considered insecure after it was publicly broken in under 24 | |
| 58 | hours. However, there is a modern implementation of DES called Triple DES where | |
| 59 | the DES method is applied three times to each data block. | |
| 60 | ||
| 61 | The main advantages to symmetric systems are the ease of use, since only one key | |
| 62 | is required for both encryption and decryption, and the simplicity of the | |
| 63 | algorithms. This helps with bulk data encryption that may unnecessarily waste | |
| 64 | time and power using asymmetric systems. | |
| 65 | ||
| 66 | However, symmetric systems have disadvantages to keep in mind. Since the key is | |
| 67 | private, it can be difficult to safely distribute keys to communication | |
| 68 | partners. Additionally, the key cannot be used to sign messages since it's | |
| 69 | necessary to keep the key private. | |
| 70 | ||
| 71 | * Asymmetric Systems (Public Key Cryptography) | |
| 72 | ||
| 73 | Asymmetric cryptography utilizes two keys within the system: a secret key that | |
| 74 | is privately-held and a public key that can be distributed freely. The | |
| 75 | interesting aspect of asymmetric cryptography is that either key can be used to | |
| 76 | encrypt the data, there's no rule that dictates which key must be used for | |
| 77 | encryption. Once one key is used to encrypt the data, only the other key can be | |
| 78 | used to decrypt the data. This means that if the private key encrypts the data, | |
| 79 | only the public key can decrypt the data. | |
| 80 | ||
| 81 | An advantage of this system is that if you successfully decrypt data using one | |
| 82 | of the keys, you can be sure of the sender since only the other key could have | |
| 83 | encrypted the data. | |
| 84 | ||
| 85 | One of the major implementations of an asymmetric system is a digital signature. | |
| 86 | A digital signature can be generated using the sender's private key, or a | |
| 87 | one-way hash function and is used to provide assurance for the integrity and | |
| 88 | authenticity of the message. A couple common message digest algorithms are | |
| 89 | SHA-256 and SHA-512, which securely compress data and produce a 128-bit message | |
| 90 | digest. | |
| 91 | ||
| 92 | It should be noted that man-in-the-middle attacks are one of the risks with | |
| 93 | digital signatures and public keys. To combat this, applications often use a | |
| 94 | public key infrastructure (PKI) to independently authenticate the validity of | |
| 95 | signatures and keys. | |
| 96 | ||
| 97 | Due to the large key size and [[https://crypto.stackexchange.com/a/591][inefficient mathematical functions]] of asymmetric | |
| 98 | encryption, elliptical curve cryptography (ECC) is often used to increase | |
| 99 | security while using fewer resources. | |
| 100 | ||
| 101 | * Applications of Cryptographic Systems | |
| 102 | ||
| 103 | There are quite a few implementations of cryptographic systems around the world. | |
| 104 | Here are a few popular examples: | |
| 105 | ||
| 106 | *Transport Layer Security (TLS):* One of the most famous cryptographic | |
| 107 | solutions created is TLS, a session-layered or connection-layered internet | |
| 108 | protocol that allows for secure communications between browsers and servers. | |
| 109 | Using handshakes, peer negotiation, and authentication allows TLS to prevent | |
| 110 | eavesdropping and malicious transformation of data. The major reason for TLS | |
| 111 | popularity is that a major vulnerability was found in the SSL protocol in 2014. | |
| 112 | Instead of SSL, TLS can be used with HTTP to form HTTPS and is the preferred | |
| 113 | method for modern web development due to its increased security. | |
| 114 | ||
| 115 | *Secure Hypertext Transfer Protocol (HTTPS):* An application layer protocol | |
| 116 | that allows for secure transport of data between servers and web clients. One of | |
| 117 | the unique parts of HTTPS is that it uses a secured port number instead of the | |
| 118 | default web port address. | |
| 119 | ||
| 120 | *Virtual Private Network (VPN):* VPNs are made to securely extend a private | |
| 121 | network across public networks by utilizing an encrypted layered tunneling | |
| 122 | protocol paired with an authentication method, such as usernames and passwords. | |
| 123 | This technology originally allowed remote employees to access their company's | |
| 124 | data but have evolved into one of the top choices for anyone who wishes to mask | |
| 125 | their sensitive personal data. | |
| 126 | ||
| 127 | *Internet Protocol Security (IPSec):* This protocol suite facilitates | |
| 128 | communication between two or more hosts or subnets by authenticating and | |
| 129 | encrypting the data packets. IPSec is used in a lot of VPNs to establish the VPN | |
| 130 | connection through the transport and tunnel mode encryption methods. IPSec | |
| 131 | encrypts just the data portion of packets in the transport methods, but it | |
| 132 | encrypts both the data and headers in the tunnel method (introducing an | |
| 133 | additional header for authentication). | |
| 134 | ||
| 135 | *Secure Shell (SSH):* SSH is another network protocol used to protect network | |
| 136 | services by authenticating users through a secure channel. This protocol is | |
| 137 | often used for command-line (shell) functions such as remote shell commands, | |
| 138 | logins, and file transfers. | |
| 139 | ||
| 140 | *Kerberos:* Developed by MIT, Kerberos is a computer-network authentication | |
| 141 | protocol that works on the basis of tickets to allow nodes communicating over a | |
| 142 | non-secure network to prove their identity to one another securely. This is most | |
| 143 | commonly used in business environments when used as the authentication and | |
| 144 | encryption method for Windows Active Directory (AD). | |
| 145 | ||
| 146 | * Cybersecurity Controls | |
| 147 | ||
| 148 | If you're someone who needs solutions on how to control risks associated with | |
| 149 | utilizing a crytograhpic system, start with a few basic controls: | |
| 150 | ||
| 151 | - *Policies:* A policy on the use of cryptographic controls for protection | |
| 152 | of information is implemented and is in accordance with organizational | |
| 153 | objectives. | |
| 154 | - *Key management:* A policy on the use, protection and lifetime of | |
| 155 | cryptographic keys is implemented through the entire application lifecycle. | |
| 156 | - *Key size:* The organization has researched the optimal key size for their | |
| 157 | purposes, considering national laws, required processing power, and | |
| 158 | longevity of the solution. | |
| 159 | - *Algorithm selection:* Implemented algorithms are sufficiently appropriate | |
| 160 | for the business of the organization, robust, and align with recommended | |
| 161 | guidelines. | |
| 162 | - *Protocol configuration:* Protocols have been reviewed and configured | |
| 163 | suitable to the purpose of the business. | |
blog/2020-03-25-session-messenger.org added +127
| @@ -0,0 +1,127 @@ | ||
| 1 | #+date: 2020-03-25 | |
| 2 | #+title: Session Private Messenger | |
| 3 | ||
| 4 | * Privacy Warning | |
| 5 | ||
| 6 | The company behind Session (Loki Foundation) is from Australia. If you didn't | |
| 7 | know, Australia has introduced [[https://parlinfo.aph.gov.au/parlInfo/download/legislation/bills/r6195_aspassed/toc_pdf/18204b01.pdf][legislation]] mandating companies comply with | |
| 8 | government requests to build backdoor access into applications. For more | |
| 9 | information, read my article on [[./2020-01-25-aes-encryption.html][AES Encryption]]. | |
| 10 | ||
| 11 | * About Session | |
| 12 | ||
| 13 | [[https://getsession.org][Session]] is a private, cross-platform messaging app from the [[https://loki.foundation][Loki Foundation]]. As | |
| 14 | someone who has spent years looking for quality alternatives to major messaging | |
| 15 | apps, I was excited when I first heard about Session. Reading through [[https://arxiv.org/pdf/2002.04609.pdf][Session's | |
| 16 | white paper]], you can learn the technologies behind the Session app. Part of the | |
| 17 | security of Session comes from the Signal protocol, which was forked as the | |
| 18 | origin of Session. | |
| 19 | ||
| 20 | #+BEGIN_QUOTE | |
| 21 | Session is an end-to-end encrypted messenger that removes sensitive metadata | |
| 22 | collection, and is designed for people who want privacy and freedom from any | |
| 23 | forms of surveillance. | |
| 24 | #+END_QUOTE | |
| 25 | ||
| 26 | In general, this app promises security through end-to-end encryption, | |
| 27 | decentralized onion routing, and private identities. The biggest change that the | |
| 28 | Loki Foundation has made to the Signal protocol is removing the need for a phone | |
| 29 | number. Instead, a random identification string is generated for any session you | |
| 30 | create. This means you can create a new session for each device if you want to, | |
| 31 | or link new devices with your ID. | |
| 32 | ||
| 33 | Since Session's website and white paper describe the details of Session's | |
| 34 | security, I'm going to focus on using the app in this post. | |
| 35 | ||
| 36 | * Features | |
| 37 | ||
| 38 | Since most people are looking for an alternative to a popular chat app, I am | |
| 39 | going to list out the features that Session has so that you are able to | |
| 40 | determine if the app would suit your needs: | |
| 41 | ||
| 42 | - Multiple device linking (via QR code or ID) | |
| 43 | - App locking via device screen lock, password, or fingerprint | |
| 44 | - Screenshot blocking | |
| 45 | - Incognito keyboard | |
| 46 | - Read receipts and typing indicators | |
| 47 | - Mobile notification customization | |
| 48 | - Old message deletion and conversation limit | |
| 49 | - Backups | |
| 50 | - Recovery phrase | |
| 51 | - Account deletion, including ID, messages, sessions, and contacts | |
| 52 | ||
| 53 | * Downloads | |
| 54 | ||
| 55 | I have tested this app on Ubuntu 19.10, Android 10, macOS Monterey, and iOS 15. | |
| 56 | All apps have worked well without many issues. | |
| 57 | ||
| 58 | Below is a brief overview of the Session app on Linux. To get this app, you'll | |
| 59 | need to go to the [[https://getsession.org/download/][Downloads]] page and click to link to the operating system | |
| 60 | you're using. | |
| 61 | ||
| 62 | For Linux, it will download an AppImage that you'll need to enable with the | |
| 63 | following command: | |
| 64 | ||
| 65 | #+BEGIN_SRC sh | |
| 66 | sudo chmod u+x session-messenger-desktop-linux-x86_64-1.0.5.AppImage | |
| 67 | #+END_SRC | |
| 68 | ||
| 69 | #+CAPTION: Session Download Options | |
| 70 | [[https://img.0x4b1d.org/blog/20200325-session-private-messenger/session_downloads.png]] | |
| 71 | ||
| 72 | * Creating an Account | |
| 73 | ||
| 74 | Once you've installed the app, simply run the app and create your unique Session | |
| 75 | ID. It will look something like this: | |
| 76 | =05af1835afdd63c947b47705867501d6373f486aa1ae05b1f2f3fcd24570eba608=. | |
| 77 | ||
| 78 | You'll need to set a display name and, optionally, a password. If you set a | |
| 79 | password, you will need to enter it every time you open the app. | |
| 80 | ||
| 81 | #+CAPTION: Session Login (Linux) | |
| 82 | [[https://img.0x4b1d.org/blog/20200325-session-private-messenger/session_linux_login.png]] | |
| 83 | ||
| 84 | #+CAPTION: Session Login (macOS) | |
| 85 | [[https://img.0x4b1d.org/blog/20200325-session-private-messenger/session_macos_login.png]] | |
| 86 | ||
| 87 | #+CAPTION: Password Authentication | |
| 88 | [[https://img.0x4b1d.org/blog/20200325-session-private-messenger/session_password_authentication.png]] | |
| 89 | ||
| 90 | * Start Messaging | |
| 91 | ||
| 92 | Once you've created your account and set up your profile details, the next step | |
| 93 | is to start messaging other people. To do so, you'll need to share your Session | |
| 94 | ID with other people. From this point, it's fairly straightforward and acts like | |
| 95 | any other messaging app, so I won't dive into much detail here. | |
| 96 | ||
| 97 | ** macOS | |
| 98 | ||
| 99 | #+CAPTION: macOS Conversations | |
| 100 | [[https://img.0x4b1d.org/blog/20200325-session-private-messenger/session_macos_conversations.png]] | |
| 101 | ||
| 102 | One key feature to note is that the desktop application now provides a helpful | |
| 103 | pop-up box explaining the process that Session uses to hide your IP address: | |
| 104 | ||
| 105 | #+CAPTION: IP Address Help Box | |
| 106 | [[https://img.0x4b1d.org/blog/20200325-session-private-messenger/session_ip.png]] | |
| 107 | ||
| 108 | ** iOS | |
| 109 | ||
| 110 | The mobile app is quite simple and effective, giving you all the standard mobile | |
| 111 | messaging options you'd expect. | |
| 112 | ||
| 113 | #+CAPTION: iOS App | |
| 114 | [[https://img.0x4b1d.org/blog/20200325-session-private-messenger/session_ios.png]] | |
| 115 | ||
| 116 | * Potential Issues | |
| 117 | ||
| 118 | I've discovered one annoying issue that would prevent from using this app | |
| 119 | regularly. On a mobile device, there have been issues with receiving messages on | |
| 120 | time. Even with battery optimization disabled and no network restrictions, | |
| 121 | Session notifications sometimes do not display until I open the app or the | |
| 122 | conversation itself and wait a few moments. This is actually one of the reasons | |
| 123 | I stopped using Signal (this seems fixed as of my updates in 2021/2022, | |
| 124 | so I wouldn't worry about this issue anymore). | |
| 125 | ||
| 126 | Looking for another messenger instead of Session? I recommend Signal, Matrix, | |
| 127 | and IRC. | |
blog/2020-05-03-homelab.org added +151
| @@ -0,0 +1,151 @@ | ||
| 1 | #+date: 2020-05-03 | |
| 2 | #+title: An Inside Look at My Homelab | |
| 3 | ||
| 4 | * What is a Homelab? | |
| 5 | ||
| 6 | Starting as a developer, I have largely stayed away from hardware-based hobbies | |
| 7 | (other than building a gaming desktop). However, as the quarantine for COVID-19 | |
| 8 | stretches out further and further, I found myself bored and in search of new | |
| 9 | hobbies. After spending the last few months browsing the [[https://www.reddit.com/r/homelab/][r/homelab]] subreddit, I | |
| 10 | decided it was time to jump in and try things out for myself. | |
| 11 | ||
| 12 | Since I am a beginner and just recently graduated from college, everything I've | |
| 13 | done so far in my homelab is fairly low-budget. | |
| 14 | ||
| 15 | * Hardware | |
| 16 | ||
| 17 | #+CAPTION: HomeLab Diagram | |
| 18 | [[https://img.0x4b1d.org/blog/20200503-homelab/homelab-min.png]] | |
| 19 | ||
| 20 | ** Raspberry Pi 4 | |
| 21 | ||
| 22 | Luckily, I had actually purchased a [[https://www.raspberrypi.org/products/raspberry-pi-4-model-b/][Raspberry Pi 4]] before the quarantine started | |
| 23 | so that I could try to keep Plex Media Center running, even while my desktop | |
| 24 | computer was turned off. I started here, using the Pi to hold Plex and Pi-hole | |
| 25 | until I grew tired with the slow performance. | |
| 26 | ||
| 27 | Here are the specifications for the Pi 4: | |
| 28 | ||
| 29 | - Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz | |
| 30 | - 4GB LPDDR4-3200 SDRAM | |
| 31 | - Gigabit Ethernet | |
| 32 | - H.265 (4kp60 decode), H264 (1080p60 decode, 1080p30 encode) | |
| 33 | - 64 GB MicroSD Card | |
| 34 | ||
| 35 | ** Dell Optiplex 5040 | |
| 36 | ||
| 37 | Since I wasn't happy with the Pi as my main server, I turned to Craigslist. I | |
| 38 | know a lot of other homelabbers use Ebay, but I can't seem to ever trust it | |
| 39 | enough to purchase items on there. So I used Craigslist and found a Dell | |
| 40 | Optiplex 5040 desktop computer on sale for $90. While this computer might be | |
| 41 | underpowered, it was one of the few computers under $100 that was available | |
| 42 | during quarantine. | |
| 43 | ||
| 44 | Here are the specifications for the Dell Optiplex 5040: | |
| 45 | ||
| 46 | - Intel Core i3 6100 | |
| 47 | - 8GB RAM DDR3 | |
| 48 | - Intel HD Graphics | |
| 49 | - Gigabit Ethernet | |
| 50 | - 500GB Hard Drive | |
| 51 | ||
| 52 | While this hardware would be awful for a work computer or a gaming rig, it | |
| 53 | turned out to be wonderful for my server purposes. The only limitation I have | |
| 54 | found so far is the CPU. The i3-6100 only has enough power for a single 4k video | |
| 55 | transcode at a time. I haven't tested more than three 1080p streams at a time, | |
| 56 | but the maximum amount of streams I've ever actually used is two. | |
| 57 | ||
| 58 | ** WD easystore 10TB & 8TB | |
| 59 | ||
| 60 | Application storage and temporary files are stored on the internal hard drive of | |
| 61 | the server, but all media files (movies, tv, games, books, etc) are stored | |
| 62 | externally on my WD easystore hard drive. Creating auto-boot configurations in | |
| 63 | the =/etc/fstab= file on my server allows the hard drives to automatically mount | |
| 64 | whenever I need to restart my server. | |
| 65 | ||
| 66 | #+BEGIN_QUOTE | |
| 67 | Update: In March 2022, I shucked the hard drives out of their external cases, | |
| 68 | put some Kapton tape on the third power pin to prevent power shutdowns, and | |
| 69 | stuck them inside my server tower using internal SATA cables. | |
| 70 | #+END_QUOTE | |
| 71 | ||
| 72 | ** Netgear Unmanaged Switch | |
| 73 | ||
| 74 | To manage all the ethernet cords used by my homelab, my desktop, and my living | |
| 75 | room media center, I purchased an 8-port gigabit ethernet switch for $50 at my | |
| 76 | local computer store. This is probably much more than I should have spent on an | |
| 77 | unmanaged switch, but I am comfortable with the choice. | |
| 78 | ||
| 79 | ** TP-Link Managed Switch | |
| 80 | ||
| 81 | Since I use the unmanaged switch to group all living room devices together, I | |
| 82 | use the managed switch to configure VLANs and secure my network. | |
| 83 | ||
| 84 | ** Arris TM1602A Modem & Sagecom Fast 5280 Router | |
| 85 | ||
| 86 | My default modem and router, provided by my ISP, are fairly standard. The Arris | |
| 87 | modem supports DOCSIS 3.0, which is something that I definitely wanted as a | |
| 88 | minimum. The Sagecom router is also standard, no fancy bells or whistles. | |
| 89 | However, it does support DHCP and DHCPv6, which is something you can use to | |
| 90 | route all household traffic through a pi-hole or firewall. | |
| 91 | ||
| 92 | ** TP-Link EAP | |
| 93 | ||
| 94 | In order to gain better control over the network, I use my own wireless access | |
| 95 | point instead of the one included in the Sagecom router above. Now I can control | |
| 96 | and organize all of my ethernet connections through the VLANs on the managed | |
| 97 | switch and wireless connections through the VLANS on the EAP. | |
| 98 | ||
| 99 | ** Generic Printer | |
| 100 | ||
| 101 | The last piece to my homelab is a standard wireless printer. Nothing special | |
| 102 | here. | |
| 103 | ||
| 104 | * Software | |
| 105 | ||
| 106 | ** Ubuntu Server 20.04 | |
| 107 | ||
| 108 | While the 20.04 version of Ubuntu was just released, I always like to experiment | |
| 109 | with new features (and I don't mind breaking my system - it just gives me more | |
| 110 | experience learning how to fix things). So, I have Ubuntu Server 20.04 installed | |
| 111 | on the Dell Optiplex server and Ubuntu Server 19.10 installed on the Raspberry | |
| 112 | Pi. Once I find an acceptable use for the Pi, I will most likely switch the | |
| 113 | operating system. | |
| 114 | ||
| 115 | ** Docker | |
| 116 | ||
| 117 | I am /very/ new to Docker, but I have had a lot of fun playing with it so far. | |
| 118 | Docker is used to create containers that can hold all the contents of a system | |
| 119 | without interfering with other software on the same system. So far, I have | |
| 120 | successfully installed pi-hole, GitLab, Gogs, and Nextcloud in containers. | |
| 121 | However, I opted to delete all of those so that I can reconfigure them more | |
| 122 | professionally at a later time. | |
| 123 | ||
| 124 | ** Plex Media Server | |
| 125 | ||
| 126 | Plex is a media center software that allows you to organize your movies, TV | |
| 127 | shows, music, photos, and videos automatically. It will even download metadata | |
| 128 | for you so that you can easily browse these collections. | |
| 129 | ||
| 130 | ** Pi-hole | |
| 131 | ||
| 132 | Pi-hole is an alternative ad-blocker that runs at the DNS level, allowing you to | |
| 133 | block traffic when it hits your network, so that you can reject any traffic you | |
| 134 | deem to be bad. Pi-hole uses blacklists and whitelists to decide which traffic | |
| 135 | block and, luckily, there are a lot of pre-made lists out there on Reddit, | |
| 136 | GitHub, etc. | |
| 137 | ||
| 138 | ** Nextcloud | |
| 139 | ||
| 140 | While I had trouble with the Docker version of Nextcloud, I was very successful | |
| 141 | when setting up the snap version. Using this, I was able to map Nextcloud to a | |
| 142 | subdomain of a domain I own in Namecheap. Additionally, Nextcloud has an | |
| 143 | integration with Let's Encrypt that allows me to issue certificates | |
| 144 | automatically to any new domain I authorize. | |
| 145 | ||
| 146 | ** Webmin | |
| 147 | ||
| 148 | To monitor my servers, and the processes running on them, I use the Webmin | |
| 149 | dashboard. This was fairly painless to set up, and I currently access it | |
| 150 | straight through the server's IP address. In the future, I will be looking to | |
| 151 | configure Webmin to use a custom domain just like Nextcloud. | |
blog/2020-05-19-customizing-ubuntu.org added +185
| @@ -0,0 +1,185 @@ | ||
| 1 | #+date: 2020-05-19 | |
| 2 | #+title: Beginner's Guide: Customizing Ubuntu | |
| 3 | ||
| 4 | * More Information | |
| 5 | ||
| 6 | For inspiration on designing your \*nix computer, check out the [[https://libredd.it/r/unixporn][r/unixporn]] | |
| 7 | subreddit! | |
| 8 | ||
| 9 | * Customizing Ubuntu | |
| 10 | ||
| 11 | New to Linux and want to add a personal touch to your machine? One of the best | |
| 12 | perks of Linux is that it is *extremely* customizable. You can change the | |
| 13 | styles of the windows, shell (status bars/docks), icons, fonts, terminals, and | |
| 14 | more. | |
| 15 | ||
| 16 | In this post, I'm going to go through customization on Ubuntu 20.04 (GNOME) | |
| 17 | since most new users tend to choose Ubuntu-based distros. If you've found a way | |
| 18 | to install Arch with i3-gaps, I'm assuming you know how to find more advanced | |
| 19 | tutorials out there on customizations. | |
| 20 | ||
| 21 | ** Required Tools | |
| 22 | ||
| 23 | #+CAPTION: Gnome Tweaks | |
| 24 | [[https://img.0x4b1d.org/blog/20200519-customizing-ubuntu/gnome-tweaks-min.png]] | |
| 25 | ||
| 26 | Ubuntu 20.04 ships with the default desktop environment [[https://www.gnome.org/][Gnome]], which includes | |
| 27 | the handy =gnome-tweaks= tool to quickly change designs. To install this, just | |
| 28 | open your terminal and enter the following command: | |
| 29 | ||
| 30 | #+BEGIN_SRC sh | |
| 31 | sudo apt install gnome-tweaks | |
| 32 | #+END_SRC | |
| 33 | ||
| 34 | After you've finished installing the tool, simply launch the Tweaks application, | |
| 35 | and you'll be able to access the various customization options available by | |
| 36 | default on Ubuntu. You might even like some of the pre-installed options. | |
| 37 | ||
| 38 | ** GNOME Application Themes | |
| 39 | ||
| 40 | To change the themes applied to applications in GNOME, you will need to change | |
| 41 | the Applications dropdown in the Appearance section of Tweaks. To add more | |
| 42 | themes, you will have to find your preferred theme online and follow the steps | |
| 43 | below to have it show up in the Tweaks tool. While you may find themes anywhere, | |
| 44 | one of the most popular sites for GNOME themes is [[https://www.gnome-look.org/][gnome-look.org]]. This website | |
| 45 | contains themes for applications, shells, icons, and cursors. | |
| 46 | ||
| 47 | Steps to import themes into Tweaks: | |
| 48 | ||
| 49 | 1. Download the theme. | |
| 50 | 2. These files are usually compressed (.zip, .tar.gz, .tar.xz), so you will need | |
| 51 | to extract the contents. This is easiest when opening the file explorer, | |
| 52 | right-clicking the compressed file, and choosing "Extract here." | |
| 53 | 3. Move the theme folder to =/usr/share/themes/=. You can do so with the | |
| 54 | following command: =sudo mv theme-folder/ /usr/share/themes/=. | |
| 55 | - Icons and cursors will be moved to the =/usr/share/icons/= folder. | |
| 56 | - Fonts will be moved to the =/usr/share/fonts/= folder Alternatively, you | |
| 57 | can move them to the =/usr/share/fonts/opentype/= or | |
| 58 | =/usr/share/fonts/opentype/= folders, if you have a specific font type. | |
| 59 | 4. Close tweaks if it is open. Re-open Tweaks and your new theme will be | |
| 60 | available in the Applications dropdown in the Appearance section of Tweaks. | |
| 61 | ||
| 62 | If the theme is not showing up after you've moved it into the themes folder, you | |
| 63 | may have uncompressed the folder into a sub-folder. You can check this by | |
| 64 | entering the theme folder and listing the contents: | |
| 65 | ||
| 66 | #+BEGIN_SRC sh | |
| 67 | cd /usr/share/themes/Mojave-Dark && ls -la | |
| 68 | #+END_SRC | |
| 69 | ||
| 70 | This is an example of what the contents of your theme folder should look like. | |
| 71 | If you just see another folder there, you should move that folder up into the | |
| 72 | =/usr/share/themes/= folder. | |
| 73 | ||
| 74 | #+BEGIN_SRC sh | |
| 75 | cinnamon COPYING gnome-shell gtk-2.0 gtk-3.0 index.theme metacity-1 plank xfwm4 | |
| 76 | #+END_SRC | |
| 77 | ||
| 78 | ** GNOME Shell Themes | |
| 79 | ||
| 80 | To change the appearance of the title bar, default dock, app menu, and other | |
| 81 | parts of the GNOME shell, you'll need to install the [[https://extensions.gnome.org/extension/19/user-themes/][user themes]] extension on | |
| 82 | [[https://extensions.gnome.org/][Gnome Extensions]]. To be able to install extensions, you will first need to | |
| 83 | install the browser extension that the website instructs you to. See this | |
| 84 | screenshot for the blue box with a link to the extension. | |
| 85 | ||
| 86 | #+CAPTION: Gnome Extensions | |
| 87 | [[https://img.0x4b1d.org/blog/20200519-customizing-ubuntu/gnome-extensions-min.png]] | |
| 88 | ||
| 89 | After the browser extension is installed, you will need to install the native | |
| 90 | host connector: | |
| 91 | ||
| 92 | #+BEGIN_SRC sh | |
| 93 | sudo apt install chrome-gnome-shell | |
| 94 | #+END_SRC | |
| 95 | ||
| 96 | Finally, you can go the [[https://extensions.gnome.org/extension/19/user-themes/][user themes]] extension page and click the installation | |
| 97 | button. This will enable the Shell option in Tweaks. Now you can move shell | |
| 98 | themes to the =/usr/share/themes= directory, using the same steps mentioned in | |
| 99 | the previous section, and enable the new theme in Tweaks. | |
| 100 | ||
| 101 | ** Icons & Cursors | |
| 102 | ||
| 103 | Icons and cursors are installed exactly the same way, so I'm grouping these | |
| 104 | together in this post. Both of these items will need to follow the same process | |
| 105 | as installing themes, except you will want to move your font folders to the | |
| 106 | =/usr/share/icons/= directory instead. | |
| 107 | ||
| 108 | ** Fonts | |
| 109 | ||
| 110 | Fonts are one of the overlooked parts of customization, but a good font can make | |
| 111 | the whole screen look different. For example, I have installed the [[https://github.com/IBM/plex/releases][IBM Plex]] | |
| 112 | fonts on my system. This follows the same process as installing themes, except | |
| 113 | you will want to move your font folders to the =/usr/share/fonts/= directory | |
| 114 | instead. | |
| 115 | ||
| 116 | ** Terminal | |
| 117 | ||
| 118 | If you spend a lot of time typing commands, you know how important the style and | |
| 119 | functionality of the terminal is. After spending a lot of time using the default | |
| 120 | GNOME terminal with [[https://en.wikipedia.org/wiki/Bash_(Unix_shell)][unix shell]], I decided to try some different options. I ended | |
| 121 | up choosing [[https://terminator-gtk3.readthedocs.io/en/latest/][Terminator]] with [[https://en.wikipedia.org/wiki/Z_shell][zsh]]. | |
| 122 | ||
| 123 | Terminator is great if you need to open multiple terminals at one time by simply | |
| 124 | right-clicking and splitting the screen into as many terminals as you want. | |
| 125 | While this project hasn't been updated in a while, [[https://github.com/gnome-terminator/terminator/issues/1][it is coming under new | |
| 126 | development]]. However, this terminal is great and I haven't experienced any | |
| 127 | errors yet. | |
| 128 | ||
| 129 | For the shell choice, I decided to choose zsh after trying it out on a fresh | |
| 130 | Manjaro installation. Zsh is great if you like to change the themes of your | |
| 131 | terminal, include icons, or add plugins. | |
| 132 | ||
| 133 | The desktop uses the [[https://github.com/zsh-users/zsh-autosuggestions][zsh-autosuggestions]] to suggest past commands as you | |
| 134 | type. In addition, it suggests corrections if you misspell a command. Lastly, it | |
| 135 | uses the =af-magic= theme, which adds dashed lines between commands, moving the | |
| 136 | user@host tag to the right side of the terminal, and changes the colors. There | |
| 137 | are plenty of plugins and themes to choose from. Just figure out what you like | |
| 138 | and add it to your =~/.zshrc= file! | |
| 139 | ||
| 140 | *** Steps to Replicate My Terminal | |
| 141 | ||
| 142 | To install zsh on Ubuntu, enter the following command into a terminal: | |
| 143 | ||
| 144 | #+BEGIN_SRC sh | |
| 145 | sudo apt install zsh | |
| 146 | #+END_SRC | |
| 147 | ||
| 148 | Then, enter the next command to activate zsh: | |
| 149 | ||
| 150 | #+BEGIN_SRC sh | |
| 151 | sudo chsh -s $(which zsh) $(whoami) | |
| 152 | #+END_SRC | |
| 153 | ||
| 154 | To install Terminator on Ubuntu: | |
| 155 | ||
| 156 | #+BEGIN_SRC sh | |
| 157 | sudo apt install terminator | |
| 158 | #+END_SRC | |
| 159 | ||
| 160 | To install Oh My Zsh on Ubuntu: | |
| 161 | ||
| 162 | #+BEGIN_SRC sh | |
| 163 | sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" | |
| 164 | #+END_SRC | |
| 165 | ||
| 166 | To install zsh-autosuggestions via Oh My Zsh: | |
| 167 | ||
| 168 | #+BEGIN_SRC sh | |
| 169 | git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions | |
| 170 | #+END_SRC | |
| 171 | ||
| 172 | Then, add the following plugin wording to your =~/.zshrc= file (the default | |
| 173 | config usually has the =git= plugin activated, so just add any other plugins to | |
| 174 | the parentheses separated by a space): | |
| 175 | ||
| 176 | #+BEGIN_SRC sh | |
| 177 | nano ~/.zshrc | |
| 178 | #+END_SRC | |
| 179 | ||
| 180 | #+BEGIN_SRC sh | |
| 181 | plugins=(git zsh-autosuggestions) | |
| 182 | #+END_SRC | |
| 183 | ||
| 184 | Finally, you need to log out of your computer and log back in so your user shell | |
| 185 | can refresh. | |
blog/2020-07-20-video-game-sales.org added +176
| @@ -0,0 +1,176 @@ | ||
| 1 | #+date: 2020-07-20 | |
| 2 | #+title: Data Exploration: Video Game Sales | |
| 3 | ||
| 4 | * Background Information | |
| 5 | ||
| 6 | This dataset (obtained from [[https://www.kaggle.com/gregorut/videogamesales/data][Kaggle]]) contains a list of video games with sales | |
| 7 | greater than 100,000 copies. It was generated by a scrape of vgchartz.com. | |
| 8 | ||
| 9 | Fields include: | |
| 10 | ||
| 11 | - Rank: Ranking of overall sales | |
| 12 | - Name: The game name | |
| 13 | - Platform: Platform of the game release (i.e. PC,PS4, etc.) | |
| 14 | - Year: Year of the game's release | |
| 15 | - Genre: Genre of the game | |
| 16 | - Publisher: Publisher of the game | |
| 17 | - NA_Sales: Sales in North America (in millions) | |
| 18 | - EU_Sales: Sales in Europe (in millions) | |
| 19 | - JP_Sales: Sales in Japan (in millions) | |
| 20 | - Other_Sales: Sales in the rest of the world (in millions) | |
| 21 | - Global_Sales: Total worldwide sales. | |
| 22 | ||
| 23 | There are 16,598 records. 2 records were dropped due to incomplete information. | |
| 24 | ||
| 25 | * Import the Data | |
| 26 | ||
| 27 | #+BEGIN_SRC python | |
| 28 | # Import the Python libraries we will be using | |
| 29 | import pandas as pd | |
| 30 | import numpy as np | |
| 31 | import seaborn as sns; sns.set() | |
| 32 | import matplotlib.pyplot as plt | |
| 33 | ||
| 34 | # Load the file using the path to the downloaded file | |
| 35 | file = r'video_game_sales.csv' | |
| 36 | df = pd.read_csv(file) | |
| 37 | df | |
| 38 | #+END_SRC | |
| 39 | ||
| 40 | #+CAPTION: Dataframe Results | |
| 41 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/01_dataframe-min.png]] | |
| 42 | ||
| 43 | * Explore the Data | |
| 44 | ||
| 45 | #+BEGIN_SRC python | |
| 46 | # With the description function, we can see the basic stats. For example, we can also see that the 'Year' column has some incomplete values. | |
| 47 | df.describe() | |
| 48 | #+END_SRC | |
| 49 | ||
| 50 | #+CAPTION: df.describe() | |
| 51 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/02_describe-min.png]] | |
| 52 | ||
| 53 | #+BEGIN_SRC python | |
| 54 | # This function shows the rows and columns of NaN values. For example, df[179,3] = nan | |
| 55 | np.where(pd.isnull(df)) | |
| 56 | ||
| 57 | (array([179, ..., 16553], dtype=int64), | |
| 58 | array([3, ..., 5], dtype=int64)) | |
| 59 | #+END_SRC | |
| 60 | ||
| 61 | * Visualize the Data | |
| 62 | ||
| 63 | #+BEGIN_SRC python | |
| 64 | # This function plots the global sales by platform | |
| 65 | sns.catplot(x='Platform', y='Global_Sales', data=df, jitter=False).set_xticklabels(rotation=90) | |
| 66 | #+END_SRC | |
| 67 | ||
| 68 | #+CAPTION: Plot of Global Sales by Platform | |
| 69 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/03_plot-min.png]] | |
| 70 | ||
| 71 | #+BEGIN_SRC python | |
| 72 | # This function plots the global sales by genre | |
| 73 | sns.catplot(x='Genre', y='Global_Sales', data=df, jitter=False).set_xticklabels(rotation=45) | |
| 74 | #+END_SRC | |
| 75 | ||
| 76 | #+CAPTION: Plot of Global Sales by Genre | |
| 77 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/04_plot-min.png]] | |
| 78 | ||
| 79 | #+BEGIN_SRC python | |
| 80 | # This function plots the global sales by year | |
| 81 | sns.lmplot(x='Year', y='Global_Sales', data=df).set_xticklabels(rotation=45) | |
| 82 | #+END_SRC | |
| 83 | ||
| 84 | #+CAPTION: Plot of Global Sales by Year | |
| 85 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/05_plot-min.png]] | |
| 86 | ||
| 87 | #+BEGIN_SRC python | |
| 88 | # This function plots four different lines to show sales from different regions. | |
| 89 | # The global sales plot line is commented-out, but can be included for comparison | |
| 90 | df2 = df.groupby('Year').sum() | |
| 91 | years = range(1980,2019) | |
| 92 | ||
| 93 | a = df2['NA_Sales'] | |
| 94 | b = df2['EU_Sales'] | |
| 95 | c = df2['JP_Sales'] | |
| 96 | d = df2['Other_Sales'] | |
| 97 | # e = df2['Global_Sales'] | |
| 98 | ||
| 99 | fig, ax = plt.subplots(figsize=(12,12)) | |
| 100 | ax.set_ylabel('Region Sales (in Millions)') | |
| 101 | ax.set_xlabel('Year') | |
| 102 | ||
| 103 | ax.plot(years, a, label='NA_Sales') | |
| 104 | ax.plot(years, b, label='EU_Sales') | |
| 105 | ax.plot(years, c, label='JP_Sales') | |
| 106 | ax.plot(years, d, label='Other_Sales') | |
| 107 | # ax.plot(years, e, label='Global_Sales') | |
| 108 | ||
| 109 | ax.legend() | |
| 110 | plt.show() | |
| 111 | #+END_SRC | |
| 112 | ||
| 113 | #+CAPTION: Plot of Regional Sales by Year | |
| 114 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/06_plot-min.png]] | |
| 115 | ||
| 116 | * Investigate Outliers | |
| 117 | ||
| 118 | #+BEGIN_SRC python | |
| 119 | # Find the game with the highest sales in North America | |
| 120 | df.loc[df['NA_Sales'].idxmax()] | |
| 121 | ||
| 122 | Rank 1 | |
| 123 | Name Wii Sports | |
| 124 | Platform Wii | |
| 125 | Year 2006 | |
| 126 | Genre Sports | |
| 127 | Publisher Nintendo | |
| 128 | NA_Sales 41.49 | |
| 129 | EU_Sales 29.02 | |
| 130 | JP_Sales 3.77 | |
| 131 | Other_Sales 8.46 | |
| 132 | Global_Sales 82.74 | |
| 133 | Name: 0, dtype: object | |
| 134 | ||
| 135 | # Explore statistics in the year 2006 (highest selling year) | |
| 136 | df3 = df[(df['Year'] == 2006)] | |
| 137 | df3.describe() | |
| 138 | #+END_SRC | |
| 139 | ||
| 140 | #+CAPTION: Descriptive Statistics of 2006 Sales | |
| 141 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/07_2006_stats-min.png]] | |
| 142 | ||
| 143 | #+BEGIN_SRC python | |
| 144 | # Plot the results of the previous dataframe (games from 2006) - we can see the year's results were largely carried by Wii Sports | |
| 145 | sns.catplot(x="Genre", y="Global_Sales", data=df3, jitter=False).set_xticklabels(rotation=45) | |
| 146 | #+END_SRC | |
| 147 | ||
| 148 | #+CAPTION: Plot of 2006 Sales | |
| 149 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/08_plot-min.png]] | |
| 150 | ||
| 151 | #+BEGIN_SRC python | |
| 152 | # We can see 4 outliers in the graph above, so let's get the top 5 games from that dataframe | |
| 153 | # The results below show that Nintendo had all top 5 games (3 on the Wii and 2 on the DS) | |
| 154 | df3.sort_values(by=['Global_Sales'], ascending=False).head(5) | |
| 155 | #+END_SRC | |
| 156 | ||
| 157 | #+CAPTION: Outliers of 2006 Sales | |
| 158 | [[https://img.0x4b1d.org/blog/20200720-data-exploration-video-game-sales/09_outliers-min.png]] | |
| 159 | ||
| 160 | * Discussion | |
| 161 | ||
| 162 | The purpose of exploring datasets is to ask questions, answer questions, and | |
| 163 | discover intelligence that can be used to inform decision-making. So, what have | |
| 164 | we found in this dataset? | |
| 165 | ||
| 166 | Today we simply explored a publicly-available dataset to see what kind of | |
| 167 | information it contained. During that exploration, we found that video game | |
| 168 | sales peaked in 2006. That peak was largely due to Nintendo, who sold the top 5 | |
| 169 | games in 2006 and has a number of games in the top-10 list for the years | |
| 170 | 1980-2020. Additionally, the top four platforms by global sales (Wii, NES, GB, | |
| 171 | DS) are owned by Nintendo. | |
| 172 | ||
| 173 | We didn't explore everything this dataset has to offer, but we can tell from a | |
| 174 | brief analysis that Nintendo seems to rule sales in the video gaming world. | |
| 175 | Further analysis could provide insight into which genres, regions, publishers, | |
| 176 | or world events are correlated with sales. | |
blog/2020-07-26-business-analysis.org added +378
| @@ -0,0 +1,378 @@ | ||
| 1 | #+date: 2020-07-26 | |
| 2 | #+title: Algorithmically Analyzing Local Businesses | |
| 3 | ||
| 4 | * Background Information | |
| 5 | ||
| 6 | This project aims to help investors learn more about a random city in order to | |
| 7 | determine optimal locations for business investments. The data used in this | |
| 8 | project was obtained using Foursquare's developer API. | |
| 9 | ||
| 10 | Fields include: | |
| 11 | ||
| 12 | - Venue Name | |
| 13 | - Venue Category | |
| 14 | - Venue Latitude | |
| 15 | - Venue Longitude | |
| 16 | ||
| 17 | There are 232 records found using the center of Lincoln as the area of interest | |
| 18 | with a radius of 10,000. | |
| 19 | ||
| 20 | * Import the Data | |
| 21 | ||
| 22 | The first step is the simplest: import the applicable libraries. We will be | |
| 23 | using the libraries below for this project. | |
| 24 | ||
| 25 | #+BEGIN_SRC python | |
| 26 | # Import the Python libraries we will be using | |
| 27 | import pandas as pd | |
| 28 | import requests | |
| 29 | import folium | |
| 30 | import math | |
| 31 | import json | |
| 32 | from pandas.io.json import json_normalize | |
| 33 | from sklearn.cluster import KMeans | |
| 34 | #+END_SRC | |
| 35 | ||
| 36 | To begin our analysis, we need to import the data for this project. The data we | |
| 37 | are using in this project comes directly from the Foursquare API. The first step | |
| 38 | is to get the latitude and longitude of the city being studied (Lincoln, NE) and | |
| 39 | setting up the folium map. | |
| 40 | ||
| 41 | #+BEGIN_SRC python | |
| 42 | # Define the latitude and longitude, then map the results | |
| 43 | latitude = 40.806862 | |
| 44 | longitude = -96.681679 | |
| 45 | map_LNK = folium.Map(location=[latitude, longitude], zoom_start=12) | |
| 46 | ||
| 47 | map_LNK | |
| 48 | #+END_SRC | |
| 49 | ||
| 50 | #+CAPTION: Blank Map | |
| 51 | [[https://img.0x4b1d.org/blog/20200726-ibm-data-science/01_blank_map-min.png]] | |
| 52 | ||
| 53 | Now that we have defined our city and created the map, we need to go get the | |
| 54 | business data. The Foursquare API will limit the results to 100 per API call, so | |
| 55 | we use our first API call below to determine the total results that Foursquare | |
| 56 | has found. Since the total results are 232, we perform the API fetching process | |
| 57 | three times (100 + 100 + 32 = 232). | |
| 58 | ||
| 59 | #+BEGIN_SRC python | |
| 60 | # Foursquare API credentials | |
| 61 | CLIENT_ID = 'your-client-id' | |
| 62 | CLIENT_SECRET = 'your-client-secret' | |
| 63 | VERSION = '20180604' | |
| 64 | ||
| 65 | # Set up the URL to fetch the first 100 results | |
| 66 | LIMIT = 100 | |
| 67 | radius = 10000 | |
| 68 | url = 'https://api.foursquare.com/v2/venues/explore?&client_id={}&client_secret={}&v={}&ll={},{}&radius={}&limit={}'.format( | |
| 69 | CLIENT_ID, | |
| 70 | CLIENT_SECRET, | |
| 71 | VERSION, | |
| 72 | latitude, | |
| 73 | longitude, | |
| 74 | radius, | |
| 75 | LIMIT) | |
| 76 | ||
| 77 | # Fetch the first 100 results | |
| 78 | results = requests.get(url).json() | |
| 79 | ||
| 80 | # Determine the total number of results needed to fetch | |
| 81 | totalResults = results['response']['totalResults'] | |
| 82 | totalResults | |
| 83 | ||
| 84 | # Set up the URL to fetch the second 100 results (101-200) | |
| 85 | LIMIT = 100 | |
| 86 | offset = 100 | |
| 87 | radius = 10000 | |
| 88 | url2 = 'https://api.foursquare.com/v2/venues/explore?&client_id={}&client_secret={}&v={}&ll={},{}&radius={}&limit={}&offset={}'.format( | |
| 89 | CLIENT_ID, | |
| 90 | CLIENT_SECRET, | |
| 91 | VERSION, | |
| 92 | latitude, | |
| 93 | longitude, | |
| 94 | radius, | |
| 95 | LIMIT, | |
| 96 | offset) | |
| 97 | ||
| 98 | # Fetch the second 100 results (101-200) | |
| 99 | results2 = requests.get(url2).json() | |
| 100 | ||
| 101 | # Set up the URL to fetch the final results (201 - 232) | |
| 102 | LIMIT = 100 | |
| 103 | offset = 200 | |
| 104 | radius = 10000 | |
| 105 | url3 = 'https://api.foursquare.com/v2/venues/explore?&client_id={}&client_secret={}&v={}&ll={},{}&radius={}&limit={}&offset={}'.format( | |
| 106 | CLIENT_ID, | |
| 107 | CLIENT_SECRET, | |
| 108 | VERSION, | |
| 109 | latitude, | |
| 110 | longitude, | |
| 111 | radius, | |
| 112 | LIMIT, | |
| 113 | offset) | |
| 114 | ||
| 115 | # Fetch the final results (201 - 232) | |
| 116 | results3 = requests.get(url3).json() | |
| 117 | #+END_SRC | |
| 118 | ||
| 119 | * Clean the Data | |
| 120 | ||
| 121 | Now that we have our data in three separate dataframes, we need to combine them | |
| 122 | into a single dataframe and make sure to reset the index so that we have a | |
| 123 | unique ID for each business. The `get_category_type` function below will pull | |
| 124 | the categories and name from each business's entry in the Foursquare data | |
| 125 | automatically. Once all the data has been labeled and combined, the results are | |
| 126 | stored in the =nearby_venues= dataframe. | |
| 127 | ||
| 128 | #+BEGIN_SRC python | |
| 129 | # This function will extract the category of the venue from the API dictionary | |
| 130 | def get_category_type(row): | |
| 131 | try: | |
| 132 | categories_list = row['categories'] | |
| 133 | except: | |
| 134 | categories_list = row['venue.categories'] | |
| 135 | ||
| 136 | if len(categories_list) == 0: | |
| 137 | return None | |
| 138 | else: | |
| 139 | return categories_list[0]['name'] | |
| 140 | ||
| 141 | # Get the first 100 venues | |
| 142 | venues = results['response']['groups'][0]['items'] | |
| 143 | nearby_venues = json_normalize(venues) | |
| 144 | ||
| 145 | # filter columns | |
| 146 | filtered_columns = ['venue.name', 'venue.categories', 'venue.location.lat', 'venue.location.lng'] | |
| 147 | nearby_venues = nearby_venues.loc[:, filtered_columns] | |
| 148 | ||
| 149 | # filter the category for each row | |
| 150 | nearby_venues['venue.categories'] = nearby_venues.apply(get_category_type, axis=1) | |
| 151 | ||
| 152 | # clean columns | |
| 153 | nearby_venues.columns = [col.split(".")[-1] for col in nearby_venues.columns] | |
| 154 | ||
| 155 | --- | |
| 156 | ||
| 157 | # Get the second 100 venues | |
| 158 | venues2 = results2['response']['groups'][0]['items'] | |
| 159 | nearby_venues2 = json_normalize(venues2) # flatten JSON | |
| 160 | ||
| 161 | # filter columns | |
| 162 | filtered_columns2 = ['venue.name', 'venue.categories', 'venue.location.lat', 'venue.location.lng'] | |
| 163 | nearby_venues2 = nearby_venues2.loc[:, filtered_columns] | |
| 164 | ||
| 165 | # filter the category for each row | |
| 166 | nearby_venues2['venue.categories'] = nearby_venues2.apply(get_category_type, axis=1) | |
| 167 | ||
| 168 | # clean columns | |
| 169 | nearby_venues2.columns = [col.split(".")[-1] for col in nearby_venues.columns] | |
| 170 | nearby_venues = nearby_venues.append(nearby_venues2) | |
| 171 | ||
| 172 | --- | |
| 173 | ||
| 174 | # Get the rest of the venues | |
| 175 | venues3 = results3['response']['groups'][0]['items'] | |
| 176 | nearby_venues3 = json_normalize(venues3) # flatten JSON | |
| 177 | ||
| 178 | # filter columns | |
| 179 | filtered_columns3 = ['venue.name', 'venue.categories', 'venue.location.lat', 'venue.location.lng'] | |
| 180 | nearby_venues3 = nearby_venues3.loc[:, filtered_columns] | |
| 181 | ||
| 182 | # filter the category for each row | |
| 183 | nearby_venues3['venue.categories'] = nearby_venues3.apply(get_category_type, axis=1) | |
| 184 | ||
| 185 | # clean columns | |
| 186 | nearby_venues3.columns = [col.split(".")[-1] for col in nearby_venues3.columns] | |
| 187 | ||
| 188 | nearby_venues = nearby_venues.append(nearby_venues3) | |
| 189 | nearby_venues = nearby_venues.reset_index(drop=True) | |
| 190 | nearby_venues | |
| 191 | #+END_SRC | |
| 192 | ||
| 193 | #+CAPTION: Clean Data | |
| 194 | [[https://img.0x4b1d.org/blog/20200726-ibm-data-science/02_clean_data-min.png]] | |
| 195 | ||
| 196 | * Visualize the Data | |
| 197 | ||
| 198 | We now have a complete, clean data set. The next step is to visualize this data | |
| 199 | onto the map we created earlier. We will be using folium's =CircleMarker()= | |
| 200 | function to do this. | |
| 201 | ||
| 202 | #+BEGIN_SRC python | |
| 203 | # add markers to map | |
| 204 | for lat, lng, name, categories in zip(nearby_venues['lat'], nearby_venues['lng'], nearby_venues['name'], nearby_venues['categories']): | |
| 205 | label = '{} ({})'.format(name, categories) | |
| 206 | label = folium.Popup(label, parse_html=True) | |
| 207 | folium.CircleMarker( | |
| 208 | [lat, lng], | |
| 209 | radius=5, | |
| 210 | popup=label, | |
| 211 | color='blue', | |
| 212 | fill=True, | |
| 213 | fill_color='#3186cc', | |
| 214 | fill_opacity=0.7, | |
| 215 | ).add_to(map_LNK) | |
| 216 | ||
| 217 | map_LNK | |
| 218 | #+END_SRC | |
| 219 | ||
| 220 |  | |
| 221 | ||
| 222 | * Clustering: /k-means/ | |
| 223 | ||
| 224 | To cluster the data, we will be using the /k-means/ algorithm. This algorithm is | |
| 225 | iterative and will automatically make sure that data points in each cluster are | |
| 226 | as close as possible to each other, while being as far as possible away from | |
| 227 | other clusters. | |
| 228 | ||
| 229 | However, we first have to figure out how many clusters to use (defined as the | |
| 230 | variable /'k'/). To do so, we will use the next two functions to calculate the | |
| 231 | sum of squares within clusters and then return the optimal number of clusters. | |
| 232 | ||
| 233 | #+BEGIN_SRC python | |
| 234 | # This function will return the sum of squares found in the data | |
| 235 | def calculate_wcss(data): | |
| 236 | wcss = [] | |
| 237 | for n in range(2, 21): | |
| 238 | kmeans = KMeans(n_clusters=n) | |
| 239 | kmeans.fit(X=data) | |
| 240 | wcss.append(kmeans.inertia_) | |
| 241 | ||
| 242 | return wcss | |
| 243 | ||
| 244 | # Drop 'str' cols so we can use k-means clustering | |
| 245 | cluster_df = nearby_venues.drop(columns=['name', 'categories']) | |
| 246 | ||
| 247 | # calculating the within clusters sum-of-squares for 19 cluster amounts | |
| 248 | sum_of_squares = calculate_wcss(cluster_df) | |
| 249 | ||
| 250 | # This function will return the optimal number of clusters | |
| 251 | def optimal_number_of_clusters(wcss): | |
| 252 | x1, y1 = 2, wcss[0] | |
| 253 | x2, y2 = 20, wcss[len(wcss)-1] | |
| 254 | ||
| 255 | distances = [] | |
| 256 | for i in range(len(wcss)): | |
| 257 | x0 = i+2 | |
| 258 | y0 = wcss[i] | |
| 259 | numerator = abs((y2-y1)*x0 - (x2-x1)*y0 + x2*y1 - y2*x1) | |
| 260 | denominator = math.sqrt((y2 - y1)**2 + (x2 - x1)**2) | |
| 261 | distances.append(numerator/denominator) | |
| 262 | ||
| 263 | return distances.index(max(distances)) + 2 | |
| 264 | ||
| 265 | # calculating the optimal number of clusters | |
| 266 | n = optimal_number_of_clusters(sum_of_squares) | |
| 267 | #+END_SRC | |
| 268 | ||
| 269 | Now that we have found that our optimal number of clusters is six, we need to | |
| 270 | perform k-means clustering. When this clustering occurs, each business is | |
| 271 | assigned a cluster number from 0 to 5 in the dataframe. | |
| 272 | ||
| 273 | #+BEGIN_SRC python | |
| 274 | # set number of clusters equal to the optimal number | |
| 275 | kclusters = n | |
| 276 | ||
| 277 | # run k-means clustering | |
| 278 | kmeans = KMeans(n_clusters=kclusters, random_state=0).fit(cluster_df) | |
| 279 | ||
| 280 | # add clustering labels to dataframe | |
| 281 | nearby_venues.insert(0, 'Cluster Labels', kmeans.labels_) | |
| 282 | #+END_SRC | |
| 283 | ||
| 284 | Success! We now have a dataframe with clean business data, along with a cluster | |
| 285 | number for each business. Now let's map the data using six different colors. | |
| 286 | ||
| 287 | #+BEGIN_SRC python | |
| 288 | # create map with clusters | |
| 289 | map_clusters = folium.Map(location=[latitude, longitude], zoom_start=12) | |
| 290 | colors = ['#0F9D58', '#DB4437', '#4285F4', '#800080', '#ce12c0', '#171717'] | |
| 291 | ||
| 292 | # add markers to the map | |
| 293 | for lat, lng, name, categories, cluster in zip(nearby_venues['lat'], nearby_venues['lng'], nearby_venues['name'], nearby_venues['categories'], nearby_venues['Cluster Labels']): | |
| 294 | label = '[{}] {} ({})'.format(cluster, name, categories) | |
| 295 | label = folium.Popup(label, parse_html=True) | |
| 296 | folium.CircleMarker( | |
| 297 | [lat, lng], | |
| 298 | radius=5, | |
| 299 | popup=label, | |
| 300 | color=colors[int(cluster)], | |
| 301 | fill=True, | |
| 302 | fill_color=colors[int(cluster)], | |
| 303 | fill_opacity=0.7).add_to(map_clusters) | |
| 304 | ||
| 305 | map_clusters | |
| 306 | #+END_SRC | |
| 307 | ||
| 308 | #+CAPTION: Clustered Map | |
| 309 | [[https://img.0x4b1d.org/blog/20200726-ibm-data-science/04_clusters-min.png]] | |
| 310 | ||
| 311 | * Investigate Clusters | |
| 312 | ||
| 313 | Now that we have figured out our clusters, let's do a little more analysis to | |
| 314 | provide more insight into the clusters. With the information below, we can see | |
| 315 | which clusters are more popular for businesses and which are less popular. The | |
| 316 | results below show us that clusters 0 through 3 are popular, while clusters 4 | |
| 317 | and 5 are not very popular at all. | |
| 318 | ||
| 319 | #+BEGIN_SRC python | |
| 320 | # Show how many venues are in each cluster | |
| 321 | color_names = ['Dark Green', 'Red', 'Blue', 'Purple', 'Pink', 'Black'] | |
| 322 | for x in range(0,6): | |
| 323 | print("Color of Cluster", x, ":", color_names[x]) | |
| 324 | print("Venues found in Cluster", x, ":", nearby_venues.loc[nearby_venues['Cluster Labels'] == x, nearby_venues.columns[:]].shape[0]) | |
| 325 | print("---") | |
| 326 | #+END_SRC | |
| 327 | ||
| 328 | #+CAPTION: Venues per Cluster | |
| 329 | [[https://img.0x4b1d.org/blog/20200726-ibm-data-science/05_venues_per_cluster-min.png]] | |
| 330 | ||
| 331 | Our last piece of analysis is to summarize the categories of businesses within | |
| 332 | each cluster. With these results, we can clearly see that restaurants, coffee | |
| 333 | shops, and grocery stores are the most popular. | |
| 334 | ||
| 335 | #+BEGIN_SRC python | |
| 336 | # Calculate how many venues there are in each category | |
| 337 | # Sort from largest to smallest | |
| 338 | temp_df = nearby_venues.drop(columns=['name', 'lat', 'lng']) | |
| 339 | ||
| 340 | cluster0_grouped = temp_df.loc[temp_df['Cluster Labels'] == 0].groupby(['categories']).count().sort_values(by='Cluster Labels', ascending=False) | |
| 341 | cluster1_grouped = temp_df.loc[temp_df['Cluster Labels'] == 1].groupby(['categories']).count().sort_values(by='Cluster Labels', ascending=False) | |
| 342 | cluster2_grouped = temp_df.loc[temp_df['Cluster Labels'] == 2].groupby(['categories']).count().sort_values(by='Cluster Labels', ascending=False) | |
| 343 | cluster3_grouped = temp_df.loc[temp_df['Cluster Labels'] == 3].groupby(['categories']).count().sort_values(by='Cluster Labels', ascending=False) | |
| 344 | cluster4_grouped = temp_df.loc[temp_df['Cluster Labels'] == 4].groupby(['categories']).count().sort_values(by='Cluster Labels', ascending=False) | |
| 345 | cluster5_grouped = temp_df.loc[temp_df['Cluster Labels'] == 5].groupby(['categories']).count().sort_values(by='Cluster Labels', ascending=False) | |
| 346 | ||
| 347 | # show how many venues there are in each cluster (> 1) | |
| 348 | with pd.option_context('display.max_rows', None, 'display.max_columns', None): | |
| 349 | print("\n\n", "Cluster 0:", "\n", cluster0_grouped.loc[cluster0_grouped['Cluster Labels'] > 1]) | |
| 350 | print("\n\n", "Cluster 1:", "\n", cluster1_grouped.loc[cluster1_grouped['Cluster Labels'] > 1]) | |
| 351 | print("\n\n", "Cluster 2:", "\n", cluster2_grouped.loc[cluster2_grouped['Cluster Labels'] > 1]) | |
| 352 | print("\n\n", "Cluster 3:", "\n", cluster3_grouped.loc[cluster3_grouped['Cluster Labels'] > 1]) | |
| 353 | print("\n\n", "Cluster 4:", "\n", cluster4_grouped.loc[cluster4_grouped['Cluster Labels'] > 1]) | |
| 354 | print("\n\n", "Cluster 5:", "\n", cluster5_grouped.loc[cluster5_grouped['Cluster Labels'] > 1]) | |
| 355 | #+END_SRC | |
| 356 | ||
| 357 | #+CAPTION: Venues per Cluster, pt. 1 | |
| 358 | [[https://img.0x4b1d.org/blog/20200726-ibm-data-science/06_categories_per_cluster_pt1-min.png]] | |
| 359 | ||
| 360 | #+CAPTION: Venues per Cluster, pt. 2 | |
| 361 | [[https://img.0x4b1d.org/blog/20200726-ibm-data-science/07_categories_per_cluster_pt2-min.png]] | |
| 362 | ||
| 363 | * Discussion | |
| 364 | ||
| 365 | In this project, we gathered location data for Lincoln, Nebraska, USA and | |
| 366 | clustered the data using the k-means algorithm in order to identify the unique | |
| 367 | clusters of businesses in Lincoln. Through these actions, we found that there | |
| 368 | are six unique business clusters in Lincoln and that two of the clusters are | |
| 369 | likely unsuitable for investors. The remaining four clusters have a variety of | |
| 370 | businesses, but are largely dominated by restaurants and grocery stores. | |
| 371 | ||
| 372 | Using this project, investors can now make more informed decisions when deciding | |
| 373 | the location and category of business in which to invest. | |
| 374 | ||
| 375 | Further studies may involve other attributes for business locations, such as | |
| 376 | population density, average wealth across the city, or crime rates. In addition, | |
| 377 | further studies may include additional location data and businesses by utilizing | |
| 378 | multiple sources, such as Google Maps and OpenStreetMap. | |
blog/2020-08-22-redirect-github-pages.org added +120
| @@ -0,0 +1,120 @@ | ||
| 1 | #+date: 2020-08-22 | |
| 2 | #+title: Redirect GitHub Pages from Subdomain to the Top-Level Domain | |
| 3 | ||
| 4 | * Short answer | |
| 5 | ||
| 6 | ** Step 1 | |
| 7 | ||
| 8 | Add a new file CNAME to your GitHub Pages repository containing only one line: | |
| 9 | your top-level domain name. E.g.: =example.com= | |
| 10 | ||
| 11 | ** Step 2 | |
| 12 | ||
| 13 | [Optional] but highly recommended | |
| 14 | ||
| 15 | 2.1: Remove all other top-level records (prefixed with @) of type A from your | |
| 16 | DNS configuration. | |
| 17 | ||
| 18 | 2.2: Remove a CNAME record for the second-level domain www if it is present. | |
| 19 | ||
| 20 | ** Step 3 | |
| 21 | ||
| 22 | Add these 5 entries to the very top of your DNS configuration: | |
| 23 | ||
| 24 | #+BEGIN_SRC txt | |
| 25 | @ A 185.199.108.153 | |
| 26 | @ A 185.199.109.153 | |
| 27 | @ A 185.199.110.153 | |
| 28 | @ A 185.199.111.153 | |
| 29 | www CNAME your_github_username.github.io. | |
| 30 | #+END_SRC | |
| 31 | ||
| 32 | Replace =your_github_username= with your actual GitHub username. | |
| 33 | ||
| 34 | ** Step 4 | |
| 35 | ||
| 36 | Wait for your DNS changes to propagate. DNS changes aren't effective | |
| 37 | immediately. They can take up to a full day to propagate. | |
| 38 | ||
| 39 | * Long answer | |
| 40 | ||
| 41 | This issue has two sides. One is the DNS configuration itself. Another one is | |
| 42 | the way GitHub Pages will forward HTTP requests. | |
| 43 | ||
| 44 | We need to know a few things to understand what GitHub is trying to say in their | |
| 45 | documentation. | |
| 46 | ||
| 47 | ** DNS Entry Types | |
| 48 | ||
| 49 | There are two types of DNS records which interest us: CNAME and A. | |
| 50 | ||
| 51 | =A= is also known as =Apex= or sometimes as =root entry=. It forwards requests | |
| 52 | to a specified fixed IP address. =CNAME= entry forwards requests to a specified | |
| 53 | URL (actual valid plain text URL, not an IP address). | |
| 54 | ||
| 55 | ** DNS Load balancing | |
| 56 | ||
| 57 | GitHub has one central URL address which accepts all DNS requests for GitHub | |
| 58 | Pages: =http://username.github.io=. That URL is resolved to different IP | |
| 59 | addresses based on your geographical location. Website hosted on GitHub Pages is | |
| 60 | a simple collection of =HTML=, =CSS= and =JS= files. GitHub distributes these | |
| 61 | files to different servers across the globe. So that when your browser sends a | |
| 62 | request from Europe, it receives data from a server in Europe. The same is valid | |
| 63 | for the requests from Asia and the USA. | |
| 64 | ||
| 65 | ** What GitHub is trying to say | |
| 66 | ||
| 67 | Since =A= records in DNS must contain IP addresses, and they must be either | |
| 68 | =185.199.108.153= or =185.199.109.153= or =185.199.110.153= or | |
| 69 | =185.199.111.153=, there is no way to forward requests to a server located | |
| 70 | somewhere in Europe or Asia. Your website hosted at GitHub Pages will be | |
| 71 | downloaded from a central GitHub Pages server. There is a minor risk that if | |
| 72 | GitHub Pages DNS servers (=x.x.x.153=) are down for some reason, all custom | |
| 73 | domains which use fixed GitHub Pages IP addresses will not be accessible (their | |
| 74 | DNS requests will not be resolvable). | |
| 75 | ||
| 76 | That is why GitHub strongly suggests to either use a second-level domain for | |
| 77 | your GitHub Pages (e.g. =blog.example.com=) or use a DNS service provider that | |
| 78 | supports a record type =ALIAS= that acts as =A= record but forwards request to a | |
| 79 | URL address (e.g. =username.github.io=) instead of a fixed IP address. | |
| 80 | ||
| 81 | ** How GitHub Pages treats HTTP requests | |
| 82 | ||
| 83 | After a DNS request for =your_github_username.github.io= is resolved into an IP | |
| 84 | address, e.g. =185.199.108.153= your browser sends an HTTP request to that | |
| 85 | server with an HTTP header =Host=. Below are =curl= examples that load the same | |
| 86 | website (these examples might not work if you are behind a proxy server): | |
| 87 | ||
| 88 | #+BEGIN_SRC sh | |
| 89 | curl --header "Host: your_github_username.github.io" http://185.199.108.153/ | |
| 90 | curl --header "Host: www.example.com" http://185.199.108.153/ | |
| 91 | curl --header "Host: example.com" http://185.199.108.153/ | |
| 92 | #+END_SRC | |
| 93 | ||
| 94 | This way GitHub Pages servers know which user website to serve. | |
| 95 | ||
| 96 | #+BEGIN_QUOTE | |
| 97 | GitHub Pages server will automatically redirect HTTP requests to the top-level | |
| 98 | domain if your =CNAME= file contains =example.com= but =www.example.com= is | |
| 99 | requested. | |
| 100 | ||
| 101 | The same is valid if your =CNAME= file contains =www.example.com= but the | |
| 102 | header =Host= in the =HTTP= request contains =example.com=. | |
| 103 | #+END_QUOTE | |
| 104 | ||
| 105 | ** Why can't I add a =CNAME= record entry that accepts a top-level request (=@=) to my DNS configuration? | |
| 106 | ||
| 107 | Quote from the GitHub Pages documentation: | |
| 108 | ||
| 109 | #+BEGIN_QUOTE | |
| 110 | Warning: Do not create a CNAME record for your custom apex domain! Doing so | |
| 111 | may cause issues with other services, such as email, on that domain. | |
| 112 | #+END_QUOTE | |
| 113 | ||
| 114 | * References: | |
| 115 | ||
| 116 | 1. [[https://docs.github.com/en/github/working-with-github-pages/configuring-a-custom-domain-for-your-github-pages-site][Setting up a custom domain with GitHub Pages]] | |
| 117 | 2. [[https://docs.github.com/en/github/working-with-github-pages/troubleshooting-custom-domains-and-github-pages][My custom domain isn't working]] | |
| 118 | 3. [[https://serverfault.com/questions/589370/cannot-access-my-github-pages-website-by-ip-address][Cannot access my GitHub Pages website by IP Address]] | |
| 119 | 4. [[https://stackoverflow.com/questions/23375422/how-do-i-set-up-github-pages-to-redirect-dns-requests-from-a-subdomain-e-g-www][How do I set up GitHub Pages to redirect DNS requests from a subdomain | |
| 120 | (e.g. www) to the top-level domain (TLD, Apex record)?]] | |
blog/2020-08-29-php-auth-flow.org added +185
| @@ -0,0 +1,185 @@ | ||
| 1 | #+date: 2020-08-29 | |
| 2 | #+title: PHP Authentication Flow | |
| 3 | ||
| 4 | * Introduction | |
| 5 | ||
| 6 | When creating websites that will allow users to create accounts, the developer | |
| 7 | always needs to consider the proper authentication flow for their app. For | |
| 8 | example, some developers will utilize an API for authentication, some will use | |
| 9 | OAuth, and some may just use their own simple database. | |
| 10 | ||
| 11 | For those using pre-built libraries, authentication may simply be a problem of | |
| 12 | copying and pasting the code from their library's documentation. For example, | |
| 13 | here's the code I use to authenticate users with the Tumblr OAuth API for my | |
| 14 | Tumblr client, Vox Populi: | |
| 15 | ||
| 16 | #+BEGIN_SRC php | |
| 17 | // Start the session | |
| 18 | session_start(); | |
| 19 | ||
| 20 | // Use my key/secret pair to create a new client connection | |
| 21 | $consumer_key = getenv('CONSUMER_KEY'); | |
| 22 | $consumer_secret = getenv('CONSUMER_SECRET'); | |
| 23 | $client = new Tumblr\API\Client($consumer_key, $consumer_secret); | |
| 24 | $requestHandler = $client->getRequestHandler(); | |
| 25 | $requestHandler->setBaseUrl('https://www.tumblr.com/'); | |
| 26 | ||
| 27 | // Check the session and cookies to see if the user is authenticated | |
| 28 | // Otherwise, send user to Tumblr authentication page and set tokens from Tumblr's response | |
| 29 | ||
| 30 | // Authenticate client | |
| 31 | $client = new Tumblr\API\Client( | |
| 32 | $consumer_key, | |
| 33 | $consumer_secret, | |
| 34 | $token, | |
| 35 | $token_secret | |
| 36 | ); | |
| 37 | #+END_SRC | |
| 38 | ||
| 39 | However, developers creating authentication flows from scratch will need to | |
| 40 | think carefully about when to make sure a web page will check the user's | |
| 41 | authenticity. | |
| 42 | ||
| 43 | In this article, we're going to look at a simple authentication flow using a | |
| 44 | MySQL database and PHP. | |
| 45 | ||
| 46 | * Creating User Accounts | |
| 47 | ||
| 48 | The beginning to any type of user authentication is to create a user account. | |
| 49 | This process can take many formats, but the simplest is to accept user input | |
| 50 | from a form (e.g., username and password) and send it over to your database. | |
| 51 | For example, here's a snippet that shows how to get username and password | |
| 52 | parameters that would come when a user submits a form to your PHP script. | |
| 53 | ||
| 54 | *Note*: Ensure that your password column is large enough to hold the hashed | |
| 55 | value (at least 60 characters or longer). | |
| 56 | ||
| 57 | #+BEGIN_SRC php | |
| 58 | // Get the values from the URL | |
| 59 | $username = $_POST['username']; | |
| 60 | $raw_password = $_POST['password']; | |
| 61 | ||
| 62 | // Hash password | |
| 63 | // password_hash() will create a random salt if one isn't provided, and this is generally the easiest and most secure approach. | |
| 64 | $password = password_hash($raw_password, PASSWORD_DEFAULT); | |
| 65 | ||
| 66 | // Save database details as variables | |
| 67 | $servername = "localhost"; | |
| 68 | $username = "username"; | |
| 69 | $password = "password"; | |
| 70 | $dbname = "myDB"; | |
| 71 | ||
| 72 | // Create connection to the database | |
| 73 | $conn = new mysqli($servername, $username, $password, $dbname); | |
| 74 | ||
| 75 | // Check connection | |
| 76 | if ($conn->connect_error) { | |
| 77 | die("Connection failed: " . $conn->connect_error); | |
| 78 | } | |
| 79 | ||
| 80 | $sql = "INSERT INTO users (username, password) | |
| 81 | VALUES ('$username', '$password')"; | |
| 82 | ||
| 83 | if ($conn->query($sql) === TRUE) { | |
| 84 | echo "New record created successfully"; | |
| 85 | } else { | |
| 86 | echo "Error: " . $sql . "<br>" . $conn->error; | |
| 87 | } | |
| 88 | ||
| 89 | $conn->close(); | |
| 90 | #+END_SRC | |
| 91 | ||
| 92 | * Validate Returning Users | |
| 93 | ||
| 94 | To be able to verify that a returning user has a valid username and password in | |
| 95 | your database is as simple as having users fill out a form and comparing their | |
| 96 | inputs to your database. | |
| 97 | ||
| 98 | #+BEGIN_SRC php | |
| 99 | // Query the database for username and password | |
| 100 | // ... | |
| 101 | ||
| 102 | if(password_verify($password_input, $hashed_password)) { | |
| 103 | // If the input password matched the hashed password in the database | |
| 104 | // Do something, log the user in. | |
| 105 | } | |
| 106 | ||
| 107 | // Else, Redirect them back to the login page. | |
| 108 | ... | |
| 109 | #+END_SRC | |
| 110 | ||
| 111 | * Storing Authentication State | |
| 112 | ||
| 113 | Once you've created the user's account, now you're ready to initialize the | |
| 114 | user's session. *You will need to do this on every page you load while the user | |
| 115 | is logged in.* To do so, simply enter the following code snippet: | |
| 116 | ||
| 117 | #+BEGIN_SRC php | |
| 118 | session_start(); | |
| 119 | #+END_SRC | |
| 120 | ||
| 121 | Once you've initialized the session, the next step is to store the session in a | |
| 122 | cookie so that you can access it later. | |
| 123 | ||
| 124 | #+BEGIN_SRC php | |
| 125 | setcookie(session_name()); | |
| 126 | #+END_SRC | |
| 127 | ||
| 128 | Now that the session name has been stored, you'll be able to check if there's an | |
| 129 | active session whenever you load a page. | |
| 130 | ||
| 131 | #+BEGIN_SRC php | |
| 132 | if(isset(session_name())) { | |
| 133 | // The session is active | |
| 134 | } | |
| 135 | #+END_SRC | |
| 136 | ||
| 137 | * Removing User Authentication | |
| 138 | ||
| 139 | The next logical step is to give your users the option to log out once they are | |
| 140 | done using your application. This can be tricky in PHP since a few of the | |
| 141 | standard ways do not always work. | |
| 142 | ||
| 143 | #+BEGIN_SRC php | |
| 144 | // Initialize the session. | |
| 145 | // If you are using session_name("something"), don't forget it now! | |
| 146 | session_start(); | |
| 147 | ||
| 148 | // Delete authentication cookies | |
| 149 | unset($_COOKIE[session_name()]); | |
| 150 | setcookie(session_name(), "", time() - 3600, "/logged-in/"); | |
| 151 | unset($_COOKIE["PHPSESSID"]); | |
| 152 | setcookie("PHPSESSID", "", time() - 3600, "/logged-in/"); | |
| 153 | ||
| 154 | // Unset all of the session variables. | |
| 155 | $_SESSION = array(); | |
| 156 | session_unset(); | |
| 157 | ||
| 158 | // If it's desired to kill the session, also delete the session cookie. | |
| 159 | // Note: This will destroy the session, and not just the session data! | |
| 160 | if (ini_get("session.use_cookies")) { | |
| 161 | $params = session_get_cookie_params(); | |
| 162 | setcookie(session_name(), '', time() - 42000, | |
| 163 | $params["path"], $params["domain"], | |
| 164 | $params["secure"], $params["httponly"] | |
| 165 | ); | |
| 166 | } | |
| 167 | ||
| 168 | // Finally, destroy the session. | |
| 169 | session_destroy(); | |
| 170 | session_write_close(); | |
| 171 | ||
| 172 | // Go back to sign-in page | |
| 173 | header('Location: https://example.com/logged-out/'); | |
| 174 | die(); | |
| 175 | #+END_SRC | |
| 176 | ||
| 177 | * Wrapping Up | |
| 178 | ||
| 179 | Now you should be ready to begin your authentication programming with PHP. You | |
| 180 | can create user accounts, create sessions for users across different pages of | |
| 181 | your site, and then destroy the user data when they're ready to leave. | |
| 182 | ||
| 183 | For more information on this subject, I recommend reading the [[https://www.php.net/][PHP | |
| 184 | Documentation]]. Specifically, you may want to look at [[https://www.php.net/manual/en/features.http-auth.php][HTTP Authentication with | |
| 185 | PHP]], [[https://www.php.net/manual/en/book.session.php][session handling]], and [[https://www.php.net/manual/en/function.hash.php][hash]]. | |
blog/2020-09-01-visual-recognition.org added +189
| @@ -0,0 +1,189 @@ | ||
| 1 | #+date: 2020-09-01 | |
| 2 | #+title: IBM Watson Visual Recognition | |
| 3 | ||
| 4 | * What is IBM Watson? | |
| 5 | ||
| 6 | If you've never heard of [[https://www.ibm.com/watson][Watson]], this service is a suite of enterprise-ready AI | |
| 7 | services, applications, and tooling provided by IBM. Watson contains quite a few | |
| 8 | useful tools for data scientists and students, including the subject of this | |
| 9 | post today: visual recognition. | |
| 10 | ||
| 11 | If you'd like to view the official documentation for the Visual Recognition API, | |
| 12 | visit the [[https://cloud.ibm.com/apidocs/visual-recognition/visual-recognition-v3?code=python][API Docs]]. | |
| 13 | ||
| 14 | * Prerequisites | |
| 15 | ||
| 16 | To be able to use Watson Visual Recognition, you'll need the following: | |
| 17 | ||
| 18 | 1. Create a free account on [[https://www.ibm.com/cloud/watson-studio][IBM Watson Studio]]. | |
| 19 | 2. Add the [[https://www.ibm.com/cloud/watson-visual-recognition][Watson Visual Recognition]] service to your IBM Watson account. | |
| 20 | 3. Get your API key and URL. To do this, first go to the [[https://dataplatform.cloud.ibm.com/home2?context=cpdaas][profile dashboard]] for | |
| 21 | your IBM account and click on the Watson Visual Recognition service you | |
| 22 | created. This will be listed in the section titled *Your services*. Then | |
| 23 | click the *Credentials* tab and open the *Auto-generated credentials* | |
| 24 | dropdown. Copy your API key and URL so that you can use them in the Python | |
| 25 | script later. | |
| 26 | 4. *[Optional]* While not required, you can also create the Jupyter Notebook for | |
| 27 | this project right inside [[https://www.ibm.com/cloud/watson-studio][Watson Studio]]. Watson Studio will save your | |
| 28 | notebooks inside an organized project and allow you to use their other | |
| 29 | integrated products, such as storage containers, AI models, documentation, | |
| 30 | external sharing, etc. | |
| 31 | ||
| 32 | * Calling the IBM Watson Visual Recognition API | |
| 33 | ||
| 34 | Okay, now let's get started. | |
| 35 | ||
| 36 | To begin, we need to install the proper Python package for IBM Watson. | |
| 37 | ||
| 38 | #+BEGIN_SRC sh | |
| 39 | pip install --upgrade --user "ibm-watson>=4.5.0" | |
| 40 | #+END_SRC | |
| 41 | ||
| 42 | Next, we need to specify the API key, version, and URL given to us when we | |
| 43 | created the Watson Visual Recognition service. | |
| 44 | ||
| 45 | #+BEGIN_SRC python | |
| 46 | apikey = "<your-apikey>" | |
| 47 | version = "2018-03-19" | |
| 48 | url = "<your-url>" | |
| 49 | #+END_SRC | |
| 50 | ||
| 51 | Now, let's import the necessary libraries and authenticate our service. | |
| 52 | ||
| 53 | #+BEGIN_SRC python | |
| 54 | import json | |
| 55 | from ibm_watson import VisualRecognitionV3 | |
| 56 | from ibm_cloud_sdk_core.authenticators import IAMAuthenticator | |
| 57 | ||
| 58 | authenticator = IAMAuthenticator(apikey) | |
| 59 | visual_recognition = VisualRecognitionV3( | |
| 60 | version=version, | |
| 61 | authenticator=authenticator | |
| 62 | ) | |
| 63 | ||
| 64 | visual_recognition.set_service_url(url) | |
| 65 | #+END_SRC | |
| 66 | ||
| 67 | *[Optional]* If you'd like to tell the API not to use any data to improve | |
| 68 | their products, set the following header. | |
| 69 | ||
| 70 | #+BEGIN_SRC python | |
| 71 | visual_recognition.set_default_headers({'x-watson-learning-opt-out': "true"}) | |
| 72 | #+END_SRC | |
| 73 | ||
| 74 | Now we have our API all set and ready to go. For this example, I'm going to | |
| 75 | include a =dict= of photos to load as we test out the API. | |
| 76 | ||
| 77 | #+BEGIN_SRC python | |
| 78 | data = [ | |
| 79 | { | |
| 80 | "title": "Grizzly Bear", | |
| 81 | "url": "https://example.com/photos/image1.jpg" | |
| 82 | }, | |
| 83 | { | |
| 84 | "title": "Nature Lake", | |
| 85 | "url": "https://example.com/photos/image2.jpg" | |
| 86 | }, | |
| 87 | { | |
| 88 | "title": "Welcome Sign", | |
| 89 | "url": "https://example.com/photos/image3.jpg" | |
| 90 | }, | |
| 91 | { | |
| 92 | "title": "Honey Badger", | |
| 93 | "url": "https://example.com/photos/image4.jpg" | |
| 94 | }, | |
| 95 | { | |
| 96 | "title": "Grand Canyon Lizard", | |
| 97 | "url": "https://example.com/photos/image5.jpg" | |
| 98 | }, | |
| 99 | { | |
| 100 | "title": "Castle", | |
| 101 | "url": "https://example.com/photos/image6.jpg" | |
| 102 | } | |
| 103 | ] | |
| 104 | #+END_SRC | |
| 105 | ||
| 106 | Now that we've set up our libraries and have the photos ready, let's create a | |
| 107 | loop to call the API for each image. The code below shows a loop that calls the | |
| 108 | URL of each image and sends it to the API, requesting results with at least 60% | |
| 109 | confidence. The results are output to the console with dotted lines separating | |
| 110 | each section. | |
| 111 | ||
| 112 | In the case of an API error, the codes and explanations are output to the | |
| 113 | console. | |
| 114 | ||
| 115 | #+BEGIN_SRC python | |
| 116 | from ibm_watson import ApiException | |
| 117 | ||
| 118 | for x in range(len(data)): | |
| 119 | try: | |
| 120 | url = data[x]["url"] | |
| 121 | images_filename = data[x]["title"] | |
| 122 | classes = visual_recognition.classify( | |
| 123 | url=url, | |
| 124 | images_filename=images_filename, | |
| 125 | threshold='0.6', | |
| 126 | owners=["IBM"]).get_result() | |
| 127 | print("-----------------------------------------------") | |
| 128 | print("Image Title: ", data[x]["title"], "\n") | |
| 129 | print("Image URL: ", data[x]["url"], "\n") | |
| 130 | classification_results = classes["images"][0]["classifiers"][0]["classes"] | |
| 131 | for result in classification_results: | |
| 132 | print(result["class"], "(", result["score"], ")") | |
| 133 | print("-----------------------------------------------") | |
| 134 | except ApiException as ex: | |
| 135 | print("Method failed with status code " + str(ex.code) + ": " + ex.message) | |
| 136 | #+END_SRC | |
| 137 | ||
| 138 | * The Results | |
| 139 | ||
| 140 | Here we can see the full result set of our function above. If you view each of | |
| 141 | the URLs that we sent to the API, you'll be able to see that it was remarkably | |
| 142 | accurate. To be fair, these are clear high-resolution, clear photos shot with a | |
| 143 | professional camera. In reality, you will most likely be processing images that | |
| 144 | are lower quality and may have a lot of noise in the photo. | |
| 145 | ||
| 146 | However, we can clearly see the benefit of being able to call this API instead | |
| 147 | of attempting to write our own image recognition function. Each of the | |
| 148 | classifications returned was a fair description of the image. | |
| 149 | ||
| 150 | If you wanted to restrict the results to those that are at least 90% confident | |
| 151 | or greater, you would simply adjust the =threshold= in the | |
| 152 | =visual_recognition.classify()= function. | |
| 153 | ||
| 154 | When your program runs, it should show the output below for each photo you | |
| 155 | provide. | |
| 156 | ||
| 157 | #+BEGIN_SRC txt | |
| 158 | ---------------------------------------------------------------- | |
| 159 | Image Title: Grizzly Bear | |
| 160 | Image URL: https://example.com/photos/image1.jpg | |
| 161 | ||
| 162 | brown bear ( 0.944 ) | |
| 163 | bear ( 1 ) | |
| 164 | carnivore ( 1 ) | |
| 165 | mammal ( 1 ) | |
| 166 | animal ( 1 ) | |
| 167 | Alaskan brown bear ( 0.759 ) | |
| 168 | greenishness color ( 0.975 ) | |
| 169 | ---------------------------------------------------------------- | |
| 170 | #+END_SRC | |
| 171 | ||
| 172 | * Discussion | |
| 173 | ||
| 174 | Now, this was a very minimal implementation of the API. We simply supplied some | |
| 175 | images and looked to see how accurate the results were. However, you could | |
| 176 | implement this type of API into many machine learning (ML) models. | |
| 177 | ||
| 178 | For example, you could be working for a company that scans their warehouses or | |
| 179 | inventory using drones. Would you want to pay employees to sit there and watch | |
| 180 | drone footage all day in order to identify or count things in the video? | |
| 181 | Probably not. Instead, you could use a classification system similar to this one | |
| 182 | in order to train your machine learning model to correctly identify items that | |
| 183 | the drones show through video. More specifically, you could have your machine | |
| 184 | learning model watch a drone fly over a field of sheep in order to count how | |
| 185 | many sheep are living in that field. | |
| 186 | ||
| 187 | There are many ways to implement machine learning functionality, but hopefully | |
| 188 | this post helped inspire some deeper thought about the tools that can help | |
| 189 | propel us further into the future of machine learning and AI. | |
blog/2020-09-22-internal-audit.org added +246
| @@ -0,0 +1,246 @@ | ||
| 1 | #+date: 2020-09-22 | |
| 2 | #+title: What is Internal Audit? | |
| 3 | ||
| 4 | #+CAPTION: Internal Audit Overview | |
| 5 | [[https://img.0x4b1d.org/blog/20200922-what-is-internal-audit/internal-audit-overview.jpg]] | |
| 6 | ||
| 7 | * Definitions | |
| 8 | ||
| 9 | One of the many reasons that Internal Audit needs such thorough explaining to | |
| 10 | non-auditors is that Internal Audit can serve many purposes, depending on the | |
| 11 | organization's size and needs. However, the Institute of Internal Auditors (IIA) | |
| 12 | defines Internal Auditing as: | |
| 13 | ||
| 14 | #+BEGIN_QUOTE | |
| 15 | Internal auditing is an independent, objective assurance and consulting activity | |
| 16 | designed to add value and improve an organization's operations. It helps an | |
| 17 | organization accomplish its objectives by bringing a systematic, disciplined | |
| 18 | approach to evaluate and improve the effectiveness of risk management, control, | |
| 19 | and governance processes. | |
| 20 | #+END_QUOTE | |
| 21 | ||
| 22 | However, this definition uses quite a few terms that aren't clear unless the | |
| 23 | reader already has a solid understanding of the auditing profession. To further | |
| 24 | explain, the following is a list of definitions that can help supplement | |
| 25 | understanding of internal auditing. | |
| 26 | ||
| 27 | ** Independent | |
| 28 | ||
| 29 | Independence is the freedom from conditions that threaten the ability of the | |
| 30 | internal audit activity to carry out internal audit responsibilities in an | |
| 31 | unbiased manner. To achieve the degree of independence necessary to effectively | |
| 32 | carry out the responsibilities of the internal audit activity, the chief audit | |
| 33 | executive has direct and unrestricted access to senior management and the board. | |
| 34 | This can be achieved through a dual-reporting relationship. Threats to | |
| 35 | independence must be managed at the individual auditor, engagement, functional, | |
| 36 | and organizational levels. | |
| 37 | ||
| 38 | ** Objective | |
| 39 | ||
| 40 | Objectivity is an unbiased mental attitude that allows internal auditors to | |
| 41 | perform engagements in such a manner that they believe in their work product and | |
| 42 | that no quality compromises are made. Objectivity requires that internal | |
| 43 | auditors do not subordinate their judgment on audit matters to others. Threats | |
| 44 | to objectivity must be managed at the individual auditor, engagement, | |
| 45 | functional, and organizational levels. | |
| 46 | ||
| 47 | ** Assurance | |
| 48 | ||
| 49 | Assurance services involve the internal auditor's objective assessment of | |
| 50 | evidence to provide opinions or conclusions regarding an entity, operation, | |
| 51 | function, process, system, or other subject matters. The internal auditor | |
| 52 | determines the nature and scope of an assurance engagement. Generally, three | |
| 53 | parties are participants in assurance services: (1) the person or group directly | |
| 54 | involved with the entity, operation, function, process, system, or other | |
| 55 | subject - (the process owner), (2) the person or group making the assessment - | |
| 56 | (the internal auditor), and (3) the person or group using the assessment - (the | |
| 57 | user). | |
| 58 | ||
| 59 | ** Consulting | |
| 60 | ||
| 61 | Consulting services are advisory in nature and are generally performed at the | |
| 62 | specific request of an engagement client. The nature and scope of the consulting | |
| 63 | engagement are subject to agreement with the engagement client. Consulting | |
| 64 | services generally involve two parties: (1) the person or group offering the | |
| 65 | advice (the internal auditor), and (2) the person or group seeking and receiving | |
| 66 | the advice (the engagement client). When performing consulting services, the | |
| 67 | internal auditor should maintain objectivity and not assume management | |
| 68 | responsibility. | |
| 69 | ||
| 70 | ** Governance, Risk Management, & Compliance (GRC) | |
| 71 | ||
| 72 | The integrated collection of capabilities that enable an organization to | |
| 73 | reliably achieve objectives, address uncertainty and act with integrity. | |
| 74 | ||
| 75 | * Audit Charter & Standards | |
| 76 | ||
| 77 | First, it's important to note that not every organization needs internal | |
| 78 | auditors. In fact, it's unwise for an organization to hire internal auditors | |
| 79 | unless they have regulatory requirements for auditing and have the capital to | |
| 80 | support the department. Internal audit is a cost center that can only affect | |
| 81 | revenue indirectly. | |
| 82 | ||
| 83 | Once an organization determines the need for internal assurance services, they | |
| 84 | will hire a Chief Audit Executive and create the audit charter. This charter is | |
| 85 | a document, approved by the company's governing body, that will define internal | |
| 86 | audit's purpose, authority, responsibility, and position within the | |
| 87 | organization. Fortunately, the IIA has model charters available to IIA members | |
| 88 | for those developing or improving their charter. | |
| 89 | ||
| 90 | Beyond the charter and organizational documents, internal auditors follow a few | |
| 91 | different standards in order to perform their job. First is the International | |
| 92 | Professional Practices Framework (IPPF) by the IIA, which is the model of | |
| 93 | standards for internal auditing. In addition, ISACA's Information Technology | |
| 94 | Assurance Framework (ITAF) helps guide auditors in reference to information | |
| 95 | technology (IT) compliance and assurance. Finally, additional standards such as | |
| 96 | FASB, GAAP, and industry-specific standards are used when performing internal | |
| 97 | audit work. | |
| 98 | ||
| 99 | * Three Lines of Defense | |
| 100 | ||
| 101 | [[https://theiia.org][The IIA]] released the original Three Lines of Defense model in 2013, but have | |
| 102 | released an updated version in 2020. Here is what the Three Lines of Defense | |
| 103 | model has historically looked like: | |
| 104 | ||
| 105 | #+CAPTION: 2013 Three Lines of Defense Model | |
| 106 | [[https://img.0x4b1d.org/blog/20200922-what-is-internal-audit/three_lines_model.png]] | |
| 107 | ||
| 108 | I won't go into depth about the changes made to the model in this article. | |
| 109 | Instead, let's take a look at the most current model. | |
| 110 | ||
| 111 | #+CAPTION: 2020 Three Lines of Defense Model | |
| 112 | [[https://img.0x4b1d.org/blog/20200922-what-is-internal-audit/updated_three_lines_model.png]] | |
| 113 | ||
| 114 | The updated model forgets the strict idea of areas performing their own | |
| 115 | functions or line of defense. Instead of talking about management, risk, and | |
| 116 | internal audit as 1-2-3, the new model creates a more fluid and cooperative | |
| 117 | model. | |
| 118 | ||
| 119 | Looking at this model from an auditing perspective shows us that auditors will | |
| 120 | need to align, communicate, and collaborate with management, including business | |
| 121 | area managers and chief officers, as well as reporting to the governing body. | |
| 122 | The governing body will instruct internal audit /functionally/ on their goals | |
| 123 | and track their progress periodically. | |
| 124 | ||
| 125 | However, the internal audit department will report /administratively/ to a chief | |
| 126 | officer in the company for the purposes of collaboration, direction, and | |
| 127 | assistance with the business. Note that in most situations, the governing body | |
| 128 | is the audit committee on the company's board of directors. | |
| 129 | ||
| 130 | The result of this structure is that internal audit is an independent and | |
| 131 | objective function that can provide assurance over the topics they audit. | |
| 132 | ||
| 133 | * Audit Process | |
| 134 | ||
| 135 | A normal audit will generally follow the same process, regardless of the topic. | |
| 136 | However, certain special projects or abnormal business areas may call for | |
| 137 | changes to the audit process. The audit process is not set in stone, it's simply | |
| 138 | a set of best practices so that audits can be performed consistently. | |
| 139 | ||
| 140 | #+CAPTION: The Internal Audit Process | |
| 141 | [[https://img.0x4b1d.org/blog/20200922-what-is-internal-audit/internal-audit-process.jpg]] | |
| 142 | ||
| 143 | While different organizations may tweak the process, it will generally follow | |
| 144 | this flow: | |
| 145 | ||
| 146 | ** 1. Risk Assessment | |
| 147 | ||
| 148 | The risk assessment part of the process has historically been performed | |
| 149 | annually, but many organizations have moved to performing this process much more | |
| 150 | frequently. In fact, some organizations are moving to an agile approach that can | |
| 151 | take new risks into the risk assessment and re-prioritize risk areas on-the-go. | |
| 152 | To perform a risk assessment, leaders in internal audit will research industry | |
| 153 | risks, consult with business leaders around the company, and perform analyses on | |
| 154 | company data. | |
| 155 | ||
| 156 | Once a risk assessment has been documented, the audit department has a | |
| 157 | prioritized list of risks that can be audited. This is usually in the form of | |
| 158 | auditable entities, such as business areas or departments. | |
| 159 | ||
| 160 | ** 2. Planning | |
| 161 | ||
| 162 | During the planning phase of an audit, auditors will meet with the business area | |
| 163 | to discuss the various processes, controls, and risks applicable to the | |
| 164 | business. This helps the auditors determine the scope limits for the audit, as | |
| 165 | well as timing and subject-matter experts. Certain documents will be created in | |
| 166 | this phase that will be used to keep the audit on-track an in-scope as it goes | |
| 167 | forward. | |
| 168 | ||
| 169 | ** 3. Testing | |
| 170 | ||
| 171 | The testing phase, also known as fieldwork or execution, is where internal | |
| 172 | auditors will take the information they've discovered and test it against | |
| 173 | regulations, industry standards, company rules, best practices, as well as | |
| 174 | validating that any processes are complete and accurate. For example, an audit | |
| 175 | of HR would most likely examine processes such as employee on-boarding, employee | |
| 176 | termination, security of personally identifiable information (PII), or the IT | |
| 177 | systems involved in these processes. Company standards would be examined and | |
| 178 | compared against how the processes are actually being performed day-to-day, as | |
| 179 | well as compared against regulations such as the Equal Employment Opportunity | |
| 180 | (EEO), American with Disabilities Act, and National Labor Relations Act. | |
| 181 | ||
| 182 | ** 4. Reporting | |
| 183 | ||
| 184 | Once all the tests have been completed, the audit will enter the reporting | |
| 185 | phase. This is when the audit team will conclude on the evidence they've | |
| 186 | collected, interviews they've held, and any opinions they've formed on the | |
| 187 | controls in place. A summary of the audit findings, conclusions, and specific | |
| 188 | recommendations are officially communicated to the client through a draft | |
| 189 | report. Clients have the opportunity to respond to the report and submit an | |
| 190 | action plan and time frame. These responses become part of the final report | |
| 191 | which is distributed to the appropriate level of administration. | |
| 192 | ||
| 193 | ** 5. Follow-Up | |
| 194 | ||
| 195 | After audits have been completed and management has formed action plans and time | |
| 196 | frames for audit issues, internal audit will follow up once that due date has | |
| 197 | arrived. In most cases, the follow-up will simply consist of a meeting to | |
| 198 | discuss how the action plan has been completed and to request documentation to | |
| 199 | prove it. | |
| 200 | ||
| 201 | * Audit Department Structure | |
| 202 | ||
| 203 | While an internal audit department is most often thought of as a team of | |
| 204 | full-time employees, there are actually many different ways in which a | |
| 205 | department can be structured. As the world becomes more digital and fast-paced, | |
| 206 | outsourcing has become a more attractive option for some organizations. Internal | |
| 207 | audit can be fully outsourced or partially outsourced, allowing for flexibility | |
| 208 | in cases where turnover is high. | |
| 209 | ||
| 210 | In addition, departments can implement a rotational model. This allows for | |
| 211 | interested employees around the organization to rotate into the internal audit | |
| 212 | department for a period of time, allowing them to obtain knowledge of risks and | |
| 213 | controls and allowing the internal audit team to obtain more business area | |
| 214 | knowledge. This program is popular in very large organizations, but | |
| 215 | organizations tend to rotate lower-level audit staff instead of managers. This | |
| 216 | helps prevent any significant knowledge loss as auditors rotate out to business | |
| 217 | areas. | |
| 218 | ||
| 219 | * Consulting | |
| 220 | ||
| 221 | Consulting is not an easy task at any organization, especially for a department | |
| 222 | that can have negative perceptions within the organization as the "compliance | |
| 223 | police." However, once an internal audit department has delivered value to | |
| 224 | organization, adding consulting to their suite of services is a smart move. In | |
| 225 | most cases, Internal Audit can insert themselves into a consulting role without | |
| 226 | affecting the process of project management at the company. This means that | |
| 227 | internal audit can add objective assurance and opinions to business areas as | |
| 228 | they develop new processes, instead of coming in periodically to audit an area | |
| 229 | and file issues that could have been fixed at the beginning. | |
| 230 | ||
| 231 | * Data Science & Data Analytics | |
| 232 | ||
| 233 | #+CAPTION: Data Science Skill Set | |
| 234 | [[https://img.0x4b1d.org/blog/20200922-what-is-internal-audit/data-science-skillset.png]] | |
| 235 | ||
| 236 | One major piece of the internal audit function in the modern world is data | |
| 237 | science. While the process is data science, most auditors will refer to anything | |
| 238 | in this realm as data analytics. Hot topics such as robotic process automation | |
| 239 | (RPA), machine learning (ML), and data mining have taken over the auditing world | |
| 240 | in recent years. These technologies have been immensely helpful with increasing | |
| 241 | the effectiveness and efficiency of auditors. | |
| 242 | ||
| 243 | For example, mundane and repetitive tasks can be automated in order for auditors | |
| 244 | to make more room in their schedules for labor-intensive work. Further, auditors | |
| 245 | will need to adapt technologies like machine learning in order to extract more | |
| 246 | value from the data they're using to form conclusions. | |
blog/2020-09-25-happiness-map.org added +217
| @@ -0,0 +1,217 @@ | ||
| 1 | #+date: 2020-09-25 | |
| 2 | #+title: Data Visualization: World Choropleth Map of Happiness | |
| 3 | ||
| 4 | * Background Information | |
| 5 | ||
| 6 | The dataset (obtained from [[https://www.kaggle.com/unsdsn/world-happiness][Kaggle]]) used in this article contains a list of | |
| 7 | countries around the world, their happiness rankings and scores, as well as | |
| 8 | other national scoring measures. | |
| 9 | ||
| 10 | Fields include: | |
| 11 | ||
| 12 | - Overall rank | |
| 13 | - Country or region | |
| 14 | - GDP per capita | |
| 15 | - Social support | |
| 16 | - Healthy life expectancy | |
| 17 | - Freedom to make life choices | |
| 18 | - Generosity | |
| 19 | - Perceptions of corruption | |
| 20 | ||
| 21 | There are 156 records. Since there are ~195 countries in the world, we can see | |
| 22 | that around 40 countries will be missing from this dataset. | |
| 23 | ||
| 24 | * Install Packages | |
| 25 | ||
| 26 | As always, run the =install= command for all packages needed to perform | |
| 27 | analysis. | |
| 28 | ||
| 29 | #+BEGIN_SRC python | |
| 30 | !pip install folium geopandas matplotlib numpy pandas | |
| 31 | #+END_SRC | |
| 32 | ||
| 33 | * Import the Data | |
| 34 | ||
| 35 | We only need a couple packages to create a choropleth map. We will use [[https://python-visualization.github.io/folium/][Folium]], | |
| 36 | which provides map visualizations in Python. We will also use geopandas and | |
| 37 | pandas to wrangle our data before we put it on a map. | |
| 38 | ||
| 39 | #+BEGIN_SRC python | |
| 40 | # Import the necessary Python packages | |
| 41 | import folium | |
| 42 | import geopandas as gpd | |
| 43 | import pandas as pd | |
| 44 | #+END_SRC | |
| 45 | ||
| 46 | To get anything to show up on a map, we need a file that will specify the | |
| 47 | boundaries of each country. Luckily, GeoJSON files exist (for free!) on the | |
| 48 | internet. To get the boundaries of every country in the world, we will use the | |
| 49 | GeoJSON link shown below. | |
| 50 | ||
| 51 | GeoPandas will take this data and load it into a dataframe so that we can easily | |
| 52 | match it to the data we're trying to analyze. Let's look at the GeoJSON | |
| 53 | dataframe: | |
| 54 | ||
| 55 | #+BEGIN_SRC python | |
| 56 | # Load the GeoJSON data with geopandas | |
| 57 | geo_data = gpd.read_file('https://raw.githubusercontent.com/datasets/geo-countries/master/data/countries.geojson') | |
| 58 | geo_data.head() | |
| 59 | #+END_SRC | |
| 60 | ||
| 61 | #+CAPTION: GeoJSON Dataframe | |
| 62 | [[https://img.0x4b1d.org/blog/20200925-world-choropleth-map/geojson_df.png]] | |
| 63 | ||
| 64 | Next, let's load the data from the Kaggle dataset. I've downloaded this file, so | |
| 65 | update the file path if you have it somewhere else. After loading, let's take a | |
| 66 | look at this dataframe: | |
| 67 | ||
| 68 | #+BEGIN_SRC python | |
| 69 | # Load the world happiness data with pandas | |
| 70 | happy_data = pd.read_csv(r'~/Downloads/world_happiness_data_2019.csv') | |
| 71 | happy_data.head() | |
| 72 | #+END_SRC | |
| 73 | ||
| 74 | #+CAPTION: Happiness Dataframe | |
| 75 | [[https://img.0x4b1d.org/blog/20200925-world-choropleth-map/happiness_df.png]] | |
| 76 | ||
| 77 | * Clean the Data | |
| 78 | ||
| 79 | Some countries need to be renamed, or they will be lost when you merge the | |
| 80 | happiness and GeoJSON dataframes. This is something I discovered when the map | |
| 81 | below showed empty countries. I searched both data frames for the missing | |
| 82 | countries to see the naming differences. Any countries that do not have records | |
| 83 | in the =happy_data= df will not show up on the map. | |
| 84 | ||
| 85 | #+BEGIN_SRC python | |
| 86 | # Rename some countries to match our GeoJSON data | |
| 87 | ||
| 88 | # Rename USA | |
| 89 | usa_index = happy_data.index[happy_data['Country or region'] == 'United States'] | |
| 90 | happy_data.at[usa_index, 'Country or region'] = 'United States of America' | |
| 91 | ||
| 92 | # Rename Tanzania | |
| 93 | tanzania_index = happy_data.index[happy_data['Country or region'] == 'Tanzania'] | |
| 94 | happy_data.at[tanzania_index, 'Country or region'] = 'United Republic of Tanzania' | |
| 95 | ||
| 96 | # Rename the Congo | |
| 97 | republic_congo_index = happy_data.index[happy_data['Country or region'] == 'Congo (Brazzaville)'] | |
| 98 | happy_data.at[republic_congo_index, 'Country or region'] = 'Republic of Congo' | |
| 99 | ||
| 100 | # Rename the DRC | |
| 101 | democratic_congo_index = happy_data.index[happy_data['Country or region'] == 'Congo (Kinshasa)'] | |
| 102 | happy_data.at[democratic_congo_index, 'Country or region'] = 'Democratic Republic of the Congo' | |
| 103 | #+END_SRC | |
| 104 | ||
| 105 | * Merge the Data | |
| 106 | ||
| 107 | Now that we have clean data, we need to merge the GeoJSON data with the | |
| 108 | happiness data. Since we've stored them both in dataframes, we just need to call | |
| 109 | the =.merge()= function. | |
| 110 | ||
| 111 | We will also rename a couple columns, just so that they're a little easier to | |
| 112 | use when we create the map. | |
| 113 | ||
| 114 | #+BEGIN_SRC python | |
| 115 | # Merge the two previous dataframes into a single geopandas dataframe | |
| 116 | merged_df = geo_data.merge(happy_data,left_on='ADMIN', right_on='Country or region') | |
| 117 | ||
| 118 | # Rename columns for ease of use | |
| 119 | merged_df = merged_df.rename(columns = {'ADMIN':'GeoJSON_Country'}) | |
| 120 | merged_df = merged_df.rename(columns = {'Country or region':'Country'}) | |
| 121 | #+END_SRC | |
| 122 | ||
| 123 | #+CAPTION: Merged Dataframe | |
| 124 | [[https://img.0x4b1d.org/blog/20200925-world-choropleth-map/merged_df.png]] | |
| 125 | ||
| 126 | * Create the Map | |
| 127 | ||
| 128 | The data is finally ready to be added to a map. The code below shows the | |
| 129 | simplest way to find the center of the map and create a Folium map object. The | |
| 130 | important part is to remember to reference the merged dataframe for our GeoJSON | |
| 131 | data and value data. The columns specify which geo data and value data to use. | |
| 132 | ||
| 133 | #+BEGIN_SRC python | |
| 134 | # Assign centroids to map | |
| 135 | x_map = merged_df.centroid.x.mean() | |
| 136 | y_map = merged_df.centroid.y.mean() | |
| 137 | print(x_map,y_map) | |
| 138 | ||
| 139 | # Creating a map object | |
| 140 | world_map = folium.Map(location=[y_map, x_map], zoom_start=2,tiles=None) | |
| 141 | folium.TileLayer('CartoDB positron',name='Dark Map',control=False).add_to(world_map) | |
| 142 | ||
| 143 | # Creating choropleth map | |
| 144 | folium.Choropleth( | |
| 145 | geo_data=merged_df, | |
| 146 | name='Choropleth', | |
| 147 | data=merged_df, | |
| 148 | columns=['Country','Overall rank'], | |
| 149 | key_on='feature.properties.Country', | |
| 150 | fill_color='YlOrRd', | |
| 151 | fill_opacity=0.6, | |
| 152 | line_opacity=0.8, | |
| 153 | legend_name='Overall happiness rank', | |
| 154 | smooth_factor=0, | |
| 155 | highlight=True | |
| 156 | ).add_to(world_map) | |
| 157 | #+END_SRC | |
| 158 | ||
| 159 | Let's look at the resulting map. | |
| 160 | ||
| 161 | #+CAPTION: Choropleth Map | |
| 162 | [[https://img.0x4b1d.org/blog/20200925-world-choropleth-map/map.png]] | |
| 163 | ||
| 164 | * Create a Tooltip on Hover | |
| 165 | ||
| 166 | Now that we have a map set up, we could stop. However, I want to add a tooltip | |
| 167 | so that I can see more information about each country. The =tooltip_data= code | |
| 168 | below will show a popup on hover with all the data fields shown. | |
| 169 | ||
| 170 | #+BEGIN_SRC python | |
| 171 | # Adding labels to map | |
| 172 | style_function = lambda x: {'fillColor': '#ffffff', | |
| 173 | 'color':'#000000', | |
| 174 | 'fillOpacity': 0.1, | |
| 175 | 'weight': 0.1} | |
| 176 | ||
| 177 | tooltip_data = folium.features.GeoJson( | |
| 178 | merged_df, | |
| 179 | style_function=style_function, | |
| 180 | control=False, | |
| 181 | tooltip=folium.features.GeoJsonTooltip( | |
| 182 | fields=['Country' | |
| 183 | ,'Overall rank' | |
| 184 | ,'Score' | |
| 185 | ,'GDP per capita' | |
| 186 | ,'Social support' | |
| 187 | ,'Healthy life expectancy' | |
| 188 | ,'Freedom to make life choices' | |
| 189 | ,'Generosity' | |
| 190 | ,'Perceptions of corruption' | |
| 191 | ], | |
| 192 | aliases=['Country: ' | |
| 193 | ,'Happiness rank: ' | |
| 194 | ,'Happiness score: ' | |
| 195 | ,'GDP per capita: ' | |
| 196 | ,'Social support: ' | |
| 197 | ,'Healthy life expectancy: ' | |
| 198 | ,'Freedom to make life choices: ' | |
| 199 | ,'Generosity: ' | |
| 200 | ,'Perceptions of corruption: ' | |
| 201 | ], | |
| 202 | style=('background-color: white; color: #333333; font-family: arial; font-size: 12px; padding: 10px;') | |
| 203 | ) | |
| 204 | ) | |
| 205 | world_map.add_child(tooltip_data) | |
| 206 | world_map.keep_in_front(tooltip_data) | |
| 207 | folium.LayerControl().add_to(world_map) | |
| 208 | ||
| 209 | # Display the map | |
| 210 | world_map | |
| 211 | #+END_SRC | |
| 212 | ||
| 213 | The final image below will show you what the tooltip looks like whenever you | |
| 214 | hover over a country. | |
| 215 | ||
| 216 | #+CAPTION: Choropleth Map Tooltip | |
| 217 | [[https://img.0x4b1d.org/blog/20200925-world-choropleth-map/tooltip_map.png]] | |
blog/2020-10-12-mediocrity.org added +111
| @@ -0,0 +1,111 @@ | ||
| 1 | #+date: 2020-10-12 | |
| 2 | #+title: On the Pursuit of Mediocrity | |
| 3 | ||
| 4 | * Perfect is the Enemy of Good | |
| 5 | ||
| 6 | As the saying goes, "the best is the enemy of the good." As we strive for | |
| 7 | perfection, we often fail to realize the implications of such an undertaking. | |
| 8 | Attempting to reach perfection is often unrealistic. Even worse, it can get in | |
| 9 | the way of achieving a good outcome. In certain situations, we try so hard to | |
| 10 | achieve the ideal solution that we have burned the bridges that would have | |
| 11 | allowed us to reach a lesser yet still superb solution. | |
| 12 | ||
| 13 | Philosophers throughout history have inspected this plight from many viewpoints. | |
| 14 | Greek mythology speaks of the [[https://en.wikipedia.org/wiki/Golden_mean_(philosophy)][golden mean]], which uses the story of Icarus to | |
| 15 | illustrate that sometimes "the middle course" is the best solution. In this | |
| 16 | story, Daedalus, a famous artist of his time, built feathered wings for himself | |
| 17 | and his son so that they might escape the clutches of King Minos. Daedalus warns | |
| 18 | his beloved son whom he loved so much to "fly the middle course", between the | |
| 19 | sea spray and the sun's heat. Icarus did not heed his father; he flew up and up | |
| 20 | until the sun melted the wax off his wings. For not heeding the middle course, | |
| 21 | he fell into the sea and drowned. | |
| 22 | ||
| 23 | More recently, management scholars have explored the [[https://en.wikipedia.org/wiki/Pareto_principle][Pareto principle]] and found | |
| 24 | that as we increase the frequency of something, or strive to perform actions to | |
| 25 | achieve some form of perfection, we run into [[https://en.wikipedia.org/wiki/Diminishing_returns][diminishing returns]]. | |
| 26 | ||
| 27 | Even further, Harold Demsetz is noted as coining the term [[https://en.wikipedia.org/wiki/Nirvana_fallacy][the Nirvana fallacy]] in | |
| 28 | 1969, which shows the fallacy of comparing actual things with unrealistic, | |
| 29 | idealized alternatives. This is another trap that we may fall into, where we are | |
| 30 | constantly thinking of the ultimate solutions to problems, when something more | |
| 31 | realistic needs to be considered. | |
| 32 | ||
| 33 | Over and over throughout history, we've found that perfection is often | |
| 34 | unrealistic and unachievable. However, we push ourselves and our peers to "give | |
| 35 | 100%" or "go the extra mile," while it may be that the better course is to give | |
| 36 | a valuable level of effort while considering the effects of further effort on | |
| 37 | the outcome. Working harder does not always help us achieve loftier goals. | |
| 38 | ||
| 39 | This has presented itself to me most recently during my time studying at my | |
| 40 | university. I was anxious and feeling the stresses of my courses, career, and | |
| 41 | personal life for quite a while, which was greatly affecting how well I was | |
| 42 | doing at school and my level of effort at work. One day, I happened to be | |
| 43 | talking to my father when he said something simple that hit home: | |
| 44 | ||
| 45 | #+BEGIN_QUOTE | |
| 46 | All you can do is show up and do your best. | |
| 47 | Worrying about the outcomes won't affect the outcome itself. | |
| 48 | #+END_QUOTE | |
| 49 | ||
| 50 | The thought was extremely straightforward and uncomplicated, yet it was | |
| 51 | something that I had lost sight of during my stress-filled years at school. Ever | |
| 52 | since then, I've found myself pausing and remembering that quote every time I | |
| 53 | get anxious or stressed. It helps to stop and think "Can I do anything to affect | |
| 54 | the outcome, or am I simply worrying over something I can't change?" | |
| 55 | ||
| 56 | * When Mediocrity Isn't Enough | |
| 57 | ||
| 58 | One problem with the philosophies presented in this post is that they are | |
| 59 | implemented far too often in situations where mediocrity simply isn't adequate. | |
| 60 | For example, let's take a look at digital user data, specifically | |
| 61 | personally-identifiable information (PII). As a cybersecurity auditor in the | |
| 62 | United States, I have found that most companies are concerned more with | |
| 63 | compliance than any actual safeguards over the privacy or protection of user | |
| 64 | data. Other than companies who have built their reputation on privacy and | |
| 65 | security, most companies will use [[https://en.wikipedia.org/wiki/Satisficing][satisficing]] as their primary decision-making | |
| 66 | strategy around user data. | |
| 67 | ||
| 68 | #+BEGIN_QUOTE | |
| 69 | Satisficing is a decision-making strategy or cognitive heuristic that entails | |
| 70 | searching through the available alternatives until an acceptability threshold | |
| 71 | is met. | |
| 72 | #+END_QUOTE | |
| 73 | ||
| 74 | This means that each decision will be met with certain possible solutions until | |
| 75 | one of the solutions meets their minimum acceptable standards. For companies | |
| 76 | that deal with user data, the minimum-acceptable standards come from three | |
| 77 | areas: | |
| 78 | ||
| 79 | 1. Laws and regulations | |
| 80 | 2. Competitive pressure | |
| 81 | 3. Risk of monetary or reputation loss | |
| 82 | ||
| 83 | Working with project management or auditing, the primary concern here is the | |
| 84 | risk of legal ramifications. Since the primary risk comes from laws and | |
| 85 | regulations, companies will require that any project that involves user data | |
| 86 | must follow all the rules of those laws so that the company can protect itself | |
| 87 | from fines or other penalties. | |
| 88 | ||
| 89 | Following this, companies will consider best practices in order to place itself | |
| 90 | in a competitive position (e.g. Google vs. Apple) and review any recent or | |
| 91 | ongoing litigation against companies regarding user data. In a perfect company, | |
| 92 | management would then consider the ethical responsibilities of their | |
| 93 | organization and discuss their responsibilities over things like | |
| 94 | personally-identifiable information. | |
| 95 | ||
| 96 | However, as we mentioned above, most companies follow the idea of satisficing, | |
| 97 | which states that they have met the minimum acceptable standards and can now | |
| 98 | move on to other decisions. Modern business culture in the United States | |
| 99 | dictates that profits are the golden measure of how well a company or manager is | |
| 100 | performing, so we often don't think about our responsibilities beyond these | |
| 101 | basic standards. | |
| 102 | ||
| 103 | Not all situations demand excellence, but I believe that applying any philosophy | |
| 104 | as a broad stroke across one's life can be a mistake. We must be able to think | |
| 105 | critically about what we are doing as we do it and ask ourselves a few | |
| 106 | questions. Have I done everything I can in this situation? Is mediocrity an | |
| 107 | acceptable outcome, or should we strive for perfection, even if we can't attain | |
| 108 | it? | |
| 109 | ||
| 110 | Taking a few moments to think critically throughout our day, as we make | |
| 111 | decisions, can have a tremendous effect on the outcomes we create. | |
blog/2020-12-27-website-redesign.org added +87
| @@ -0,0 +1,87 @@ | ||
| 1 | #+date: 2020-12-27 | |
| 2 | #+title: Redesigning My Website: The 5 KB Result | |
| 3 | ||
| 4 | * A Brief History | |
| 5 | ||
| 6 | As a form of continuous learning and entertainment, I've been running a handful | |
| 7 | of websites since 2016 when I took my first programming courses in college. I | |
| 8 | maintain one main website, the place I consider the official website to | |
| 9 | represent me. Under this site, I have a handful of subdirectories and | |
| 10 | subdomains. | |
| 11 | ||
| 12 | One of the parts I've enjoyed the most about web development is the aspect of | |
| 13 | designing an identity for a web page and working to find exciting ways to | |
| 14 | display the site's content. Inevitably, this means I've changed the designs for | |
| 15 | my websites more times than I could possibly count. Since I don't really host | |
| 16 | anything on my main webpage that's vital, it allows me the freedom to change | |
| 17 | things as inspiration strikes. | |
| 18 | ||
| 19 | Historically, I've relied on core utilities for spacing, components, and layouts | |
| 20 | from [[https://getbootstrap.com][Bootstrap]] and added custom CSS for fonts, accents, colors, and other | |
| 21 | items. I also tend to create sites with no border radius on items, visible | |
| 22 | borders, and content that takes up the entire screen (using whitespace inside | |
| 23 | components instead of whitespace around my components). | |
| 24 | ||
| 25 | * The Redesign Process | |
| 26 | ||
| 27 | About a week ago, I found myself wishing for a new design yet again. The prior | |
| 28 | design was largely inspired by IBM's [[https://www.carbondesignsystem.com][Carbon Design System]] and relied on jQuery, | |
| 29 | Bootstrap, along with some compressed [[https://en.wikipedia.org/wiki/WebP][.webp]] images. | |
| 30 | ||
| 31 | To anyone who knows my preferences toward web design - and even in my personal | |
| 32 | life - it should be no surprise that I immediately started looking for | |
| 33 | inspiration on minimalism. While there are some decent minimalistic designs on | |
| 34 | sites like [[https://dribbble.com/search/shots/popular/web-design?q=minimalism][Dribbble]], people seem to mostly discuss [[https://brutalist-web.design][brutalist web design]] when you | |
| 35 | ask about minimalism. While brutalist web design doesn't have to be minimal, it | |
| 36 | often is. | |
| 37 | ||
| 38 | I suppose, in a way, I did create a brutalist website since my HTML is semantic | |
| 39 | and accessible, hyperlinks are colored and underlined, and all native browser | |
| 40 | functions like scrolling and the back button work as expected. However, I didn't | |
| 41 | think about brutalism while designing these sites. | |
| 42 | ||
| 43 | The new design followed a simple design process. I walked through the screens on | |
| 44 | my blog and asked myself: "Is this element necessary for a user?" This allowed | |
| 45 | me to first start by removing all javascript, which had the sole purpose of | |
| 46 | allowing users to open a collapsed navbar on mobile. Replacing the collapsible | |
| 47 | navbar allowed me to remove both jQuery and Bootstrap's javascript. | |
| 48 | ||
| 49 | Next, I removed things like author names (since I'm literally the only person | |
| 50 | who will ever write on this site), multiple ways to click on a blog post card, | |
| 51 | blog post descriptions, and the scroll-to-top button. It also helped to move all | |
| 52 | categories to a single page, rather than have each category on its own page. | |
| 53 | ||
| 54 | The final big piece to finish the "[[https://en.wikipedia.org/wiki/Marie_Kondo#KonMari_method][KonMari]]"-like part of my process was to | |
| 55 | remove Bootstrap CSS in its entirety. However, this meant pulling out a few very | |
| 56 | useful classes, such as `.img-fluid` and the default font stacks to keep in my | |
| 57 | custom CSS. | |
| 58 | ||
| 59 | After removing all the unnecessary pieces, I was finally able to reorganize my | |
| 60 | content and add a very small amount of custom CSS to make everything pretty. | |
| 61 | This took a brief amount of time, effectively just consisting of me | |
| 62 | converting =<div>= tags into things like =<ul>= lists and choosing accent | |
| 63 | colors. | |
| 64 | ||
| 65 | * The Results | |
| 66 | ||
| 67 | ** Reflection | |
| 68 | ||
| 69 | So, what did all of this reorganizing do to my webpages? Well, first, my | |
| 70 | websites are now *ridiculously fast*. Since the prior designs were also minimal | |
| 71 | and didn't have many images, they measured up in Firefox's Network Monitor | |
| 72 | around 300 KB - 600KB. After making the changes, my main site is at 5 KB | |
| 73 | transferred (22 KB total), and my blog is at 6.5 KB transferred (13 KB | |
| 74 | total). *That means the redesigned pages are less than 2% the size of the old | |
| 75 | designs.* | |
| 76 | ||
| 77 | Google Lighthouse ranks the new webpage as 100 in performance, accessibility, | |
| 78 | and best practices, with SEO at 92 since they think tap targets are not sized | |
| 79 | appropriately for mobile users. First contextual paints of the pages are under | |
| 80 | 0.8 seconds with 0 ms of blocking time. However, the blog subdomain ranks at 100 | |
| 81 | for all four categories! First contextual paints of the blog homepage are under | |
| 82 | 1.0 seconds with 0 ms of blocking time, due to the fact that the CSS for my blog | |
| 83 | is within a separate CSS file, and the CSS for my main website is simply | |
| 84 | embedded in the HTML file. | |
| 85 | ||
| 86 | Now that everything is complete, I can confidently say I'm happy with the result | |
| 87 | and proud to look at the fastest set of websites I've created so far. | |
blog/2020-12-28-neon-drive.org added +83
| @@ -0,0 +1,83 @@ | ||
| 1 | #+date: 2020-12-28 | |
| 2 | #+title: Neon Drive: A Nostalgic 80s Arcade Racing Game | |
| 3 | ||
| 4 | * Game Description | |
| 5 | ||
| 6 | [[https://store.steampowered.com/app/433910/Neon_Drive/][Neon Drive]] presents itself as a simple arcade-style game inspired by the arcade | |
| 7 | race games of the 1980s, yet it has managed to take up hours of my life without | |
| 8 | much effort. The game description, directly from the Steam page, is intriguing | |
| 9 | enough to entice anyone who's been looking for a good arcade racing game: | |
| 10 | ||
| 11 | #+BEGIN_QUOTE | |
| 12 | Neon Drive is a slick retro-futuristic arcade game that will make your brain | |
| 13 | melt. You've been warned. From beautiful cityscapes and ocean roads to | |
| 14 | exploding enemy spaceships, Neon Drive has it all. | |
| 15 | #+END_QUOTE | |
| 16 | ||
| 17 | * Gameplay | |
| 18 | ||
| 19 | The game holds true to the [[https://en.wikipedia.org/wiki/Retrofuturism][retro-futurism]] style, including chrome female robots, | |
| 20 | pixelated arcade machines, and [[https://teddit.net/r/outrun/][outrun]] aesthetics. | |
| 21 | ||
| 22 | Each level of the game is shown as a separate arcade machine. Each arcade | |
| 23 | machine lets you play on Normal, Hard, Insane, Practice, and Free Run. To beat | |
| 24 | each arcade, you must reach the end of the level without crashing your car into | |
| 25 | the various obstacles on the course. Basic levels let you move left or right to | |
| 26 | avoid blocks in the road. Later levels put you through other tests, such as | |
| 27 | dodging traffic or blasting asteroids. | |
| 28 | ||
| 29 | The game uses synthwave music to keep you on track to make the correct moves by | |
| 30 | timing the beats of the songs to the correct moves on the screen. It reminds me | |
| 31 | of the early Guitar Hero games, as well as mobile apps like VOEZ - repetition | |
| 32 | and staying on-beat is the only way to win. | |
| 33 | ||
| 34 | * In-Game Screenshots | |
| 35 | ||
| 36 | Taking a look at the main menu, you can see that Neon Drive plays into every | |
| 37 | stereotype you can think of around retro-futuristic, synthwave arcades (in a | |
| 38 | good way). | |
| 39 | ||
| 40 | #+CAPTION: Neon Drive Menu | |
| 41 | [[https://img.0x4b1d.org/blog/20201228-neon-drive/neon_drive_menu.png]] | |
| 42 | ||
| 43 | Once you get into the first level, we see that the choice of car fits right in | |
| 44 | with the stereotypical cars of the 80s, like the [[https://en.wikipedia.org/wiki/DMC_DeLorean][DeLorean]] or the [[https://en.wikipedia.org/wiki/Ferrari_F40][Ferrari | |
| 45 | F40]]. Each new level comes with new color schemes and cars, so you should never | |
| 46 | get tired of the aesthetic. | |
| 47 | ||
| 48 | #+CAPTION: Neon Drive Race | |
| 49 | [[https://img.0x4b1d.org/blog/20201228-neon-drive/neon_drive_race.png]] | |
| 50 | ||
| 51 | Personally, I love the orange and blue colors used in level 2: | |
| 52 | ||
| 53 | #+CAPTION: Level 2 | |
| 54 | [[https://img.0x4b1d.org/blog/20201228-neon-drive/neon_drive_level_2.png]] | |
| 55 | ||
| 56 | If you're the competitive type and getting 100% on all arcade machines isn't | |
| 57 | enough, there are leaderboards for the regular part of the game, and the | |
| 58 | endurance game mode. | |
| 59 | ||
| 60 | #+CAPTION: Leaderboard | |
| 61 | [[https://img.0x4b1d.org/blog/20201228-neon-drive/neon_drive_leaderboard.png]] | |
| 62 | ||
| 63 | * Other Suggestions | |
| 64 | ||
| 65 | Neon Drive sits nicely within the well-founded cult genre of Outrun. Other games | |
| 66 | that I've enjoyed in this same spectrum are: | |
| 67 | ||
| 68 | - [[https://store.steampowered.com/app/233270/Far_Cry_3__Blood_Dragon/][Far Cry 3: Blood Dragon]] | |
| 69 | - [[https://store.steampowered.com/app/1239690/Retrowave/][Retrowave]] | |
| 70 | - [[https://store.steampowered.com/app/732810/Slipstream/][Slipstream]] | |
| 71 | ||
| 72 | Although these games aren't necessarily in the same genre, they do have aspects | |
| 73 | that place them close enough to interest gamers that enjoyed Neon Drive: | |
| 74 | ||
| 75 | - [[https://store.steampowered.com/app/311800/Black_Ice/][Black Ice]] | |
| 76 | - [[https://store.steampowered.com/app/746850/Cloudpunk/][Cloudpunk]] | |
| 77 | - [[https://store.steampowered.com/app/1222680/Need_for_Speed_Heat/][Need for Speed: Heat]] | |
| 78 | - [[https://store.steampowered.com/app/1019310/VirtuaVerse/][VirtuaVerse]] | |
| 79 | ||
| 80 | Of course, if all you really care about is the arcade aspect of these games, you | |
| 81 | can check out the [[https://store.steampowered.com/app/400020/Atari_Vault/][Atari Vault]] or any of the other classic games sold on Steam by | |
| 82 | companies like Namco, Atari. For something like Nintendo, you'd have to settle | |
| 83 | for buying used classic consoles or delve into the world of emulation. | |
blog/2020-12-29-zork.org added +80
| @@ -0,0 +1,80 @@ | ||
| 1 | #+date: 2020-12-29 | |
| 2 | #+title: Zork: Let's Explore a Classic | |
| 3 | ||
| 4 | * Download (Free) | |
| 5 | ||
| 6 | Before we get into the game itself, you should know that you can download Zork | |
| 7 | for free from Infocom's [[http://infocom-if.org/downloads/downloads.html][download page]]. So feel free to boot it up and take a | |
| 8 | ride back to the 1980s with this masterpiece. | |
| 9 | ||
| 10 | * Game Description | |
| 11 | ||
| 12 | Zork is an interactive, text-based computer game originally released in 1980. | |
| 13 | This series, split into three separate games, introduced a robust and | |
| 14 | sophisticated text parser to gamers. People were largely used to the simple | |
| 15 | commands used in the popular game [[https://en.wikipedia.org/wiki/Colossal_Cave_Adventure][Colossal Cave Adventure]], but Zork allowed | |
| 16 | users to send more complex commands that included prepositions and conjunctions. | |
| 17 | ||
| 18 | Zork tracks your score as you explore the map, find tools, and collect trophy | |
| 19 | items (e.g., a jewel-encrusted egg). When you place your trophy items in the | |
| 20 | trophy case found in the Living Room area, you gain score points. Collecting the | |
| 21 | Twenty Treasures of Zork and placing them within the trophy case wins the | |
| 22 | game. However, you must explore the map, solve puzzles, and avoid being eaten by | |
| 23 | a grue to collect these treasures. | |
| 24 | ||
| 25 | * The Map | |
| 26 | ||
| 27 | Since Zork is a vast and complex game, it helps to have a map as you explore and | |
| 28 | collect your trophies. However, if you want to play the game as it was truly | |
| 29 | intended, you should try to play it without using the map. | |
| 30 | ||
| 31 | #+CAPTION: Zork Map | |
| 32 | [[https://img.0x4b1d.org/blog/20201229-zork/zork_map.png]] | |
| 33 | ||
| 34 | /[[https://www.filfre.net/2012/01/exploring-zork-part-1/][Map Source]]/ | |
| 35 | ||
| 36 | * In-Game Screenshots | |
| 37 | ||
| 38 | After playing the game (for the first time ever) for several weeks around 2014, | |
| 39 | I was finally able to beat the game with some online help to find the last | |
| 40 | couple items. As I was writing this post, I installed the game again to grab | |
| 41 | some screenshots to show off the true glory of this game. As noted in [[https://www.filfre.net/2012/01/exploring-zork-part-1/][Jimmy | |
| 42 | Maher's playthrough]], the original Zork games looked quite a bit different due to | |
| 43 | the older hardware of computers like the Apple II and multiple bug fixes that | |
| 44 | Infocom pushed out after the game's initial release. My play-through uses the | |
| 45 | [[https://store.steampowered.com/app/570580/Zork_Anthology/][Zork Anthology]] version, which utilizes DOSBox on Windows. | |
| 46 | ||
| 47 | The first screenshot here shows the introductory information, which doesn't | |
| 48 | include instructions of any kind for the player. If you haven't played text | |
| 49 | adventures before, try to use simple commands like "go west," "look around," or | |
| 50 | "hit troll with elvish sword." | |
| 51 | ||
| 52 | #+CAPTION: Zork Screen, pt. 1 | |
| 53 | [[https://img.0x4b1d.org/blog/20201229-zork/zork_01.png]] | |
| 54 | ||
| 55 | In this second screenshot, we see the player has entered the house and found the | |
| 56 | trophy case in the living room. The lantern and sword in this room allow the | |
| 57 | player to explore dark areas and attack enemies. If you don't use the lantern, | |
| 58 | you won't be able to see anything in dark areas, and you may be eaten by a grue. | |
| 59 | ||
| 60 | #+CAPTION: Zork Screen, pt. 2 | |
| 61 | [[https://img.0x4b1d.org/blog/20201229-zork/zork_02.png]] | |
| 62 | ||
| 63 | Finally, we see that the player has found the first treasure: a jewel-encrusted | |
| 64 | egg. These treasures can be taken back to the house and placed in the trophy | |
| 65 | case or carried until you feel like you want to put things away. | |
| 66 | ||
| 67 | #+CAPTION: Zork Screen, pt 3. | |
| 68 | [[https://img.0x4b1d.org/blog/20201229-zork/zork_03.png]] | |
| 69 | ||
| 70 | * Conclusion | |
| 71 | ||
| 72 | It's been quite a few years since I first played Zork, but I clearly remember | |
| 73 | the late nights and bloodshot eyes that helped me find all the treasures. This | |
| 74 | game is well worth the time and effort, even though the text-based aspect may be | |
| 75 | off-putting to gamers who didn't have to grow up playing games without graphics. | |
| 76 | However, I believe that the strategy and skills learned in early video games | |
| 77 | like Zork can actually help you, even when playing newer games. | |
| 78 | ||
| 79 | If you do decide to play Zork, you can download Zork I, II, and III from | |
| 80 | Infocom's [[http://infocom-if.org/downloads/downloads.html][download page]] for free or search the internet for an online version. | |
blog/2021-01-01-seum.org added +84
| @@ -0,0 +1,84 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-01-01 | |
| 3 | title = "SEUM: Speedrunners from Hell" | |
| 4 | description = "A video game review for SEUM: Speedrunners from Hell, an incredibly fast-paced mashup of a puzzle game and a free-running game." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Game Description | |
| 8 | ||
| 9 | [SEUM: Speedrunners from Hell](https://store.steampowered.com/app/457210/SEUM_Speedrunners_from_Hell/) | |
| 10 | is an incredibly fast-paced mashup of a puzzle game and a free-running game. | |
| 11 | Created by [Pine Studio](https://pinestudio.co) in early 2016 and first released | |
| 12 | as a | |
| 13 | [demo on GameJolt](https://gamejolt.com/games/seum-speedrunners-from-hell-demo/154868), | |
| 14 | this game was successfully green-lit on Steam and has amassed a cult | |
| 15 | following on multiple consoles. | |
| 16 | ||
| 17 | Here's the game description straight from the developers: | |
| 18 | ||
| 19 | > Like a bastard child of Quake 3 and Super Meat Boy, SEUM: Speedrunners from | |
| 20 | > Hell is truly hardcore and focuses on speed and fast reaction. | |
| 21 | ||
| 22 | ## Story | |
| 23 | ||
| 24 | SEUM does a great job setting the stage when you open the game for the first | |
| 25 | time, playing an animated video in the form of comic book images. You see Marty, | |
| 26 | the protagonist, sitting around drinking his beer as Satan busts down the door, | |
| 27 | cuts his hand off, and steals all his beer and bacon. As Satan is leaving, Marty | |
| 28 | whips a vinyl record across the room and cuts off one of Satan's hands. This | |
| 29 | hand is what allows you to use all the powers in the game. | |
| 30 | ||
| 31 | Check out the screenshot below for one of the first panels of the storyline: | |
| 32 | ||
| 33 |  | |
| 34 | ||
| 35 | ## Gameplay | |
| 36 | ||
| 37 | To accomplish each level, you will need to get to the final blue portal under a | |
| 38 | certain time limit. You can beat a level by getting to the portal before the | |
| 39 | time meter runs out or "Dominate" a level by beating it within a shorter time | |
| 40 | limit (noted by a bright red/orange color in the time meter). | |
| 41 | ||
| 42 | The portal is usually set behind locked gates, protected by dangerous defenses, | |
| 43 | or in hard-to-reach places. To reach the portal, you'll need to go through any | |
| 44 | existing orange portals, light all yellow beacons, avoid things like fireballs | |
| 45 | and blades, or use any satanic power orbs lying around. These special abilities | |
| 46 | include: | |
| 47 | ||
| 48 | - Gravity | |
| 49 | - Teleport | |
| 50 | - Rewind | |
| 51 | - Spawn platform | |
| 52 | - Roar (DLC) | |
| 53 | - Rocket (DLC) | |
| 54 | - Shadow world (DLC) | |
| 55 | ||
| 56 | For the main storyline, there are nine floors to beat. Each floor contains nine | |
| 57 | regular levels, one boss level, and one bonus level; although you don't | |
| 58 | technically need to beat all levels to advance to the next floor. | |
| 59 | ||
| 60 |  | |
| 61 | ||
| 62 | ## In-Game Screenshots | |
| 63 | ||
| 64 | The main menu gives you plenty of options for tuning your system, playing main | |
| 65 | levels, playing the DLC, or exploring the other game modes. | |
| 66 | ||
| 67 |  | |
| 68 | ||
| 69 | Once you enter a level, you can look around and strategize before starting. | |
| 70 | Clicking any button will start the menu, and you'll have to restart if you die. | |
| 71 | One of the great things about SEUM is that it has great keyboard shortcuts. You | |
| 72 | can quickly smash `R` to restart the level or `M` to return to the level menu. | |
| 73 | ||
| 74 |  | |
| 75 | ||
| 76 | Once you're able to win a level, you'll see the high scores and can watch | |
| 77 | replays of the best scores. | |
| 78 | ||
| 79 |  | |
| 80 | ||
| 81 | Each regular level contains a beer in a disguised location that may take some | |
| 82 | out-of-the-box thinking. | |
| 83 | ||
| 84 |  | |
blog/2021-01-04-fediverse.org added +106
| @@ -0,0 +1,106 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-01-04 | |
| 3 | title = "A Simple Guide to the Fediverse" | |
| 4 | description = "A simple guide to the fediverse, a federated universe of servers commonly used for sharing content, like social media." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## What is the Fediverse? | |
| 9 | ||
| 10 | The fediverse is a federated universe of servers commonly used for sharing | |
| 11 | content, like social media. So, instead of having to rely on a single | |
| 12 | organization to run the server (e.g. Facebook), the fediverse is a giant | |
| 13 | collection of servers across the world, owned by many people and organizations. | |
| 14 | ||
| 15 | Take a look at this depiction of a federated network. | |
| 16 | Each server in this photo is owned and run by different | |
| 17 | administrators/owners. | |
| 18 | Federated networks are best explained as email servers: you have an email | |
| 19 | account that exists on a server (e.g. Outlook), your friend has an account on | |
| 20 | a different server (e.g. GMail), and another friend has an account on a | |
| 21 | third server (e.g. ProtonMail). | |
| 22 | All three of you can talk and communicate back and forth without having to | |
| 23 | be on the same server. | |
| 24 | However, responsible email admins are there to set rules and control the | |
| 25 | traffic going in/out of the server. | |
| 26 | ||
| 27 |  | |
| 28 | ||
| 29 | The main objective of this architecture is to decentralize the control within | |
| 30 | the internet connections. For example, if you run your own Mastodon instance, | |
| 31 | you and your users can't be censored or impacted in any way by authorities of | |
| 32 | another Mastodon instance. Some users have praised these features due to recent | |
| 33 | criticism of popular social media websites that may be over-censoring their | |
| 34 | users. | |
| 35 | ||
| 36 | This strategy is great for making sure control of the social web isn't | |
| 37 | controlled by a single organization, but it also has some downsides. If I create | |
| 38 | a Mastodon instance and get a ton of users to sign up, I can shut the server | |
| 39 | down at any time. That means you're at risk of losing the content you've created | |
| 40 | unless you back it up, or the server backs it up for you. Also, depending on the | |
| 41 | software used (e.g. Mastodon, Pixelfed, etc.), censorship may still be an issue | |
| 42 | if the server admins decide they want to censor their users. Now, censorship | |
| 43 | isn't always a bad thing and can even benefit the community as a whole, but | |
| 44 | you'll want to determine which servers align with your idea of proper | |
| 45 | censorship. | |
| 46 | ||
| 47 | However, these are risks that we take when we sign up for any online platform. | |
| 48 | Whatever your reason is for trying out federated social networks, they are part | |
| 49 | of the future of the internet. However, the popularity of these services is | |
| 50 | still yet to be determined, especially with the increased difficulty | |
| 51 | understanding and signing up for these platforms. Perhaps increased regulation | |
| 52 | and litigation against current social media sites will push more users into the | |
| 53 | fediverse. | |
| 54 | ||
| 55 | ## Federated Alternatives to Popular Sites | |
| 56 | ||
| 57 | The list below is a small guide that will show you federated alternatives to | |
| 58 | current popular websites. | |
| 59 | There are many more out there, so go and explore: you might just find the | |
| 60 | perfect home. | |
| 61 | ||
| 62 | ||
| 63 | ||
| 64 | - [Lemmy](https://lemmy.ml/instances) | |
| 65 | ||
| 66 | ### Twitter/Facebook/Tumblr | |
| 67 | ||
| 68 | - [Mastodon](https://joinmastodon.org) | |
| 69 | - [Diaspora](https://diasporafoundation.org) | |
| 70 | - [Friendica](https://friendi.ca) | |
| 71 | - [GNU Social](https://gnusocial.network) | |
| 72 | - [Pleroma](https://pleroma.social) | |
| 73 | ||
| 74 | ||
| 75 | ||
| 76 | - [Pixelfed](https://pixelfed.org) | |
| 77 | ||
| 78 | ### Slack/Discord | |
| 79 | ||
| 80 | - [Matrix](https://element.io) | |
| 81 | ||
| 82 | ### Youtube/Vimeo | |
| 83 | ||
| 84 | - [Peertube](https://joinpeertube.org) | |
| 85 | ||
| 86 | ### Spotify/Soundcloud | |
| 87 | ||
| 88 | - [Funkwhale](https://funkwhale.audio) | |
| 89 | ||
| 90 | ### Podcasting | |
| 91 | ||
| 92 | - [Pubcast](https://pubcast.pub) | |
| 93 | ||
| 94 | ### Medium/Blogger | |
| 95 | ||
| 96 | - [WriteFreely](https://writefreely.org) | |
| 97 | ||
| 98 | ## Get Started | |
| 99 | ||
| 100 | The best way to get started is to simply sign up and learn as you go. If you're | |
| 101 | comfortable signing up through a Mastodon, Pleroma, or Friendica server, here is | |
| 102 | [a list of themed servers](https://fediverse.party/en/portal/servers) to choose | |
| 103 | from. If you're looking for something else, try a web search for a federated | |
| 104 | alternative to your favorite sites. | |
| 105 | ||
| 106 | Find a server that focuses on your passions and start there! | |
blog/2021-01-07-ufw.org added +220
| @@ -0,0 +1,220 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-01-07 | |
| 3 | title = "Secure Your Network with the Uncomplicated Firewall" | |
| 4 | description = "Learn how to use the UFW, one of the simplest and most widely available firewalls on Linux." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Uncomplicated Firewall | |
| 9 | ||
| 10 | Uncomplicated Firewall, also known as ufw, is a convenient and beginner-friendly | |
| 11 | way to enforce OS-level firewall rules. For those who are hosting servers or any | |
| 12 | device that is accessible to the world (i.e., by public IP or domain name), it's | |
| 13 | critical that a firewall is properly implemented and active. | |
| 14 | ||
| 15 | Ufw is available by default in all Ubuntu installations after 8.04 LTS. For | |
| 16 | other distributions, you can look to install ufw or check if there are | |
| 17 | alternative firewalls installed already. There are usually alternatives | |
| 18 | available, such as Fedora's `firewall` and the package available on most | |
| 19 | distributions: `iptables`. Ufw is considered a beginner-friendly front-end to | |
| 20 | iptables. | |
| 21 | ||
| 22 | [Gufw](https://gufw.org) is available as a graphical user interface (GUI) | |
| 23 | application for users who are uncomfortable setting up a firewall through a | |
| 24 | terminal. | |
| 25 | ||
| 26 |  | |
| 27 | ||
| 28 | ## Getting Help | |
| 29 | ||
| 30 | If you need help figuring out commands, remember that you can run the `--help` | |
| 31 | flag to get a list of options. | |
| 32 | ||
| 33 | ```sh | |
| 34 | sudo ufw --help | |
| 35 | ``` | |
| 36 | ||
| 37 | ## Set Default State | |
| 38 | ||
| 39 | The proper way to run a firewall is to set a strict default state and slowly | |
| 40 | open up ports that you want to allow. This helps prevent anything malicious from | |
| 41 | slipping through the cracks. The following command prevents all incoming traffic | |
| 42 | (other than the rules we specify later), but you can also set this for outgoing | |
| 43 | connections, if necessary. | |
| 44 | ||
| 45 | ```sh | |
| 46 | sudo ufw default deny incoming | |
| 47 | ``` | |
| 48 | ||
| 49 | You should also allow outgoing traffic if you want to allow the device to | |
| 50 | communicate back to you or other parties. For example, media servers like Plex | |
| 51 | need to be able to send out data related to streaming the media. | |
| 52 | ||
| 53 | ```sh | |
| 54 | sudo ufw default allow outgoing | |
| 55 | ``` | |
| 56 | ||
| 57 | ## Adding Port Rules | |
| 58 | ||
| 59 | Now that we've disabled all incoming traffic by default, we need to open up some | |
| 60 | ports (or else no traffic would be able to come in). If you need to be able to | |
| 61 | `ssh` into the machine, you'll need to open up port 22. | |
| 62 | ||
| 63 | ```sh | |
| 64 | sudo ufw allow 22 | |
| 65 | ``` | |
| 66 | ||
| 67 | You can also issue more restrictive rules. The following rule will allow `ssh` | |
| 68 | connections only from machines on the local subnet. | |
| 69 | ||
| 70 | ```sh | |
| 71 | sudo ufw allow proto tcp from 192.168.0.0/24 to any port 22 | |
| 72 | ``` | |
| 73 | ||
| 74 | If you need to set a rule that isn't tcp, just append your connection type to | |
| 75 | the end of the rule. | |
| 76 | ||
| 77 | ```sh | |
| 78 | sudo ufw allow 1900/udp | |
| 79 | ``` | |
| 80 | ||
| 81 | ## Enable ufw | |
| 82 | ||
| 83 | Now that the firewall is configured and ready to go, you can enable the | |
| 84 | firewall. | |
| 85 | ||
| 86 | ```sh | |
| 87 | sudo ufw enable | |
| 88 | ``` | |
| 89 | ||
| 90 | A restart may be required for the firewall to begin operating. | |
| 91 | ||
| 92 | ```sh | |
| 93 | sudo reboot now | |
| 94 | ``` | |
| 95 | ||
| 96 | ## Checking Status | |
| 97 | ||
| 98 | Now that the firewall is enabled, let's check and see what the rules look like. | |
| 99 | ||
| 100 | ```sh | |
| 101 | sudo ufw status numbered | |
| 102 | ``` | |
| 103 | ||
| 104 | ```txt | |
| 105 | Status: active | |
| 106 | ||
| 107 | To Action From | |
| 108 | -- ------ ---- | |
| 109 | [ 1] 22 ALLOW IN Anywhere | |
| 110 | [ 2] 22 (v6) ALLOW IN Anywhere (v6) | |
| 111 | ``` | |
| 112 | ||
| 113 | ## Deleting Rules | |
| 114 | ||
| 115 | If you need to delete a rule, you need to know the number associated with that | |
| 116 | rule. | |
| 117 | Let's delete the first rule in the table above. | |
| 118 | You'll be asked to confirm the deletion as part of this process. | |
| 119 | ||
| 120 | ```sh | |
| 121 | sudo ufw delete 1 | |
| 122 | ``` | |
| 123 | ||
| 124 | ## Managing App Rules | |
| 125 | ||
| 126 | Luckily, there's a convenient way for installed applications to create files | |
| 127 | that ufw can easily implement so that you don't have to search and find which | |
| 128 | ports your application requires. To see if your device has any applications with | |
| 129 | pre-installed ufw rules, execute the following command: | |
| 130 | ||
| 131 | ```sh | |
| 132 | sudo ufw app list | |
| 133 | ``` | |
| 134 | ||
| 135 | The results should look something like this: | |
| 136 | ||
| 137 | ```txt | |
| 138 | Available applications: | |
| 139 | OpenSSH | |
| 140 | Samba | |
| 141 | plexmediaserver | |
| 142 | plexmediaserver-all | |
| 143 | plexmediaserver-dlna | |
| 144 | ``` | |
| 145 | ||
| 146 | If you want to get more information on a specific app rule, use the `info` | |
| 147 | command. | |
| 148 | ||
| 149 | ```sh | |
| 150 | sudo ufw app info plexmediaserver-dlna | |
| 151 | ``` | |
| 152 | ||
| 153 | You'll get a blurb of info back like this: | |
| 154 | ||
| 155 | ```txt | |
| 156 | Profile: plexmediaserver-dlna | |
| 157 | Title: Plex Media Server (DLNA) | |
| 158 | Description: The Plex Media Server (additional DLNA capability only) | |
| 159 | ||
| 160 | Ports: | |
| 161 | 1900/udp | |
| 162 | 32469/tcp | |
| 163 | ``` | |
| 164 | ||
| 165 | You can add or delete app rules the same way that you'd add or delete specific | |
| 166 | port rules. | |
| 167 | ||
| 168 | ```sh | |
| 169 | sudo ufw allow plexmediaserver-dlna | |
| 170 | ``` | |
| 171 | ||
| 172 | ```sh | |
| 173 | sudo ufw delete RULE|NUM | |
| 174 | ``` | |
| 175 | ||
| 176 | ## Creating App Rules | |
| 177 | ||
| 178 | If you'd like to create you own app rule, you'll need to create a file in the | |
| 179 | `/etc/ufw/applications.d` directory. Within the file you create, you need to | |
| 180 | make sure the content is properly formatted. | |
| 181 | ||
| 182 | For example, here are the contents my `plexmediaserver` file, which creates | |
| 183 | three distinct app rules for ufw: | |
| 184 | ||
| 185 | ```config | |
| 186 | [plexmediaserver] | |
| 187 | title=Plex Media Server (Standard) | |
| 188 | description=The Plex Media Server | |
| 189 | ports=32400/tcp|3005/tcp|5353/udp|8324/tcp|32410:32414/udp | |
| 190 | ||
| 191 | [plexmediaserver-dlna] | |
| 192 | title=Plex Media Server (DLNA) | |
| 193 | description=The Plex Media Server (additional DLNA capability only) | |
| 194 | ports=1900/udp|32469/tcp | |
| 195 | ||
| 196 | [plexmediaserver-all] | |
| 197 | title=Plex Media Server (Standard + DLNA) | |
| 198 | description=The Plex Media Server (with additional DLNA capability) | |
| 199 | ports=32400/tcp|3005/tcp|5353/udp|8324/tcp|32410:32414/udp|1900/udp|32469/tcp | |
| 200 | ``` | |
| 201 | ||
| 202 | So, if I wanted to create a custom app rule called "mycustomrule," I'd create a | |
| 203 | file and add my content like this: | |
| 204 | ||
| 205 | ```sh | |
| 206 | sudo nano /etc/ufw/applications.d/mycustomrule | |
| 207 | ``` | |
| 208 | ||
| 209 | ```config | |
| 210 | [mycustomrule] | |
| 211 | title=My Custom Rule | |
| 212 | description=This is a temporary ufw app rule. | |
| 213 | ports=88/tcp|9100/udp | |
| 214 | ``` | |
| 215 | ||
| 216 | Then, I would just enable this rule in ufw. | |
| 217 | ||
| 218 | ```sh | |
| 219 | sudo ufw allow mycustomrule | |
| 220 | ``` | |
blog/2021-02-19-macos.org added +205
| @@ -0,0 +1,205 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-02-19 | |
| 3 | title = "macOS: Testing Out A New OS" | |
| 4 | description = "My personal experience with macOS, from the perspective of a Linux junkie." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Diving into macOS | |
| 9 | ||
| 10 | After spending nearly 15 years working with Windows and 8 years on Linux, I | |
| 11 | have experienced macOS for the first time. | |
| 12 | By chance, my spouse happened to buy a new MacBook and gifted me their 2013 | |
| 13 | model. | |
| 14 | Of course, I still consider my Linux desktop to be my daily driver and keep | |
| 15 | Windows around for gaming needs, but over the past week I've found myself using | |
| 16 | the MacBook more and more for things that don't require gaming specs or advanced | |
| 17 | dev tools. | |
| 18 | ||
| 19 | ## Initial Thoughts | |
| 20 | ||
| 21 | Before I move on to the technical aspects of my set-up, I want to take some time | |
| 22 | and express my thoughts on the overall OS. | |
| 23 | ||
| 24 |  | |
| 25 | ||
| 26 | As expected, the initial computer setup is a breeze with Mac's guided GUI | |
| 27 | installer. | |
| 28 | ||
| 29 | The desktop itself reminds me of GNOME more than anything else I've seen: even | |
| 30 | Pantheon from [ElementaryOS](https://elementary.io/), which people commonly | |
| 31 | refer to as the closest Linux distro to macOS. The desktop toolbar is great and | |
| 32 | far surpasses the utility of the GNOME toolbar due to the fact that the | |
| 33 | extensions and icons _actually work_. I launch macOS and immediately see my | |
| 34 | shortcuts for Tresorit, Bitwarden, and Mullvad pop up as the computer loads. | |
| 35 | ||
| 36 | Even further, the app dock is very useful and will be yet another familiarity | |
| 37 | for GNOME users. I know many people like panels instead of docks, but I've | |
| 38 | always found docks to have a more pleasing UI. However, I had to disable the | |
| 39 | "Show recent applications in Dock" preference; I can't stand items taking up | |
| 40 | precious screen space if I'm not currently using them. On that same note, it's | |
| 41 | taking me some time to get use to the fact that I have to manually quit an app | |
| 42 | or else it will still stay open/active in the dock, even if I've closed out all | |
| 43 | windows for that app (e.g. Firefox). | |
| 44 | ||
| 45 | Overall, I'm having a lot of fun and for users who spend a large majority of | |
| 46 | their time performing basic tasks like web browsing, writing, watching media, | |
| 47 | etc., macOS is a fantastic option. | |
| 48 | ||
| 49 | The rest of this post explains the technicalities of how I set up my CLI | |
| 50 | environment to make me feel more at-home, similar to the environments I set up | |
| 51 | on Fedora, Ubuntu, etc. | |
| 52 | ||
| 53 | ## Making it Feel Like Home | |
| 54 | ||
| 55 | If you're someone who uses Linux primarily, no doubt your first thought when | |
| 56 | booting macOS will be the same as mine was: "Where is the terminal and how do I | |
| 57 | set up my favorite utilities?" | |
| 58 | ||
| 59 | Luckily, macOS hasn't completely hidden away the development tools from the | |
| 60 | average user. You can easily find the Terminal app in the Launchpad area, but | |
| 61 | it's probably not what you're used to. I was surprised (and happy) to see that | |
| 62 | the default shell is `zsh`, the shell I use on all of my Linux distros. However, | |
| 63 | the commands are not the same - even the ones you may think are native to the | |
| 64 | shell. Commands like `dir` do not exist, so other native commands like `ls -la` | |
| 65 | or `pwd` are more useful here. | |
| 66 | ||
| 67 | With only a few minutes of installing and tweaking a few packages, I was able to | |
| 68 | recreate a terminal environment that I feel very comfortable using. See the | |
| 69 | image below for a preview of the iTerm2 app with a split view between my macOS | |
| 70 | desktop shell and an SSH session into my server. | |
| 71 | ||
| 72 |  | |
| 73 | ||
| 74 | ## Xcode | |
| 75 | ||
| 76 | My first step was to search the web for any hints on how to get `zsh` back up to | |
| 77 | the state I like, with extensions, themes, etc. My first step was to install the | |
| 78 | CLI tools for [Xcode](https://developer.apple.com/xcode/), Apple's suite of | |
| 79 | development tools. | |
| 80 | ||
| 81 | ```sh | |
| 82 | sudo xcode-select -r | |
| 83 | ``` | |
| 84 | ||
| 85 | ```sh | |
| 86 | sudo xcode-select --install | |
| 87 | ``` | |
| 88 | ||
| 89 | ## Homebrew | |
| 90 | ||
| 91 | Next up is to install [Homebrew](https://brew.sh), a nifty package manager for | |
| 92 | macOS. | |
| 93 | ||
| 94 | ```sh | |
| 95 | /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" | |
| 96 | ``` | |
| 97 | ||
| 98 | I ran into a permission error when installing Homebrew: | |
| 99 | ||
| 100 | ```sh | |
| 101 | Error: Failed to link all completions, docs and manpages: | |
| 102 | Permission denied @ rb_file_s_symlink - (../../../Homebrew/completions/zsh/_brew, /usr/local/share/zsh/site-functions/_brew) | |
| 103 | Failed during: /usr/local/bin/brew update --force --quiet | |
| 104 | ``` | |
| 105 | ||
| 106 | I found that the following permission modification worked like a charm. However, | |
| 107 | I noted that some users online discussed the fact that this solution may not | |
| 108 | work if your system has multiple users who use Homebrew. | |
| 109 | ||
| 110 | ```sh | |
| 111 | sudo chown -R $(whoami) $(brew --prefix)/* | |
| 112 | ``` | |
| 113 | ||
| 114 | Next up is to ensure Homebrew is updated and cleaned. | |
| 115 | ||
| 116 | ```sh | |
| 117 | brew update | |
| 118 | ``` | |
| 119 | ||
| 120 | ```sh | |
| 121 | brew cleanup | |
| 122 | ``` | |
| 123 | ||
| 124 | ## iTerm2 | |
| 125 | ||
| 126 | Now that I've installed the basic utilities for development, I moved onto | |
| 127 | installing iTerm2, a much better terminal than the default. | |
| 128 | ||
| 129 | ```sh | |
| 130 | brew install --cask iterm2 | |
| 131 | ``` | |
| 132 | ||
| 133 | I also used the `Make iTerm2 Default Term` and `Install Shell Integration` | |
| 134 | options in the iTerm2 application menu to make sure I don't run into any issues | |
| 135 | later on with different terminals. | |
| 136 | ||
| 137 | We will also install `zsh` so we can use it in iTerm2. | |
| 138 | ||
| 139 | ```sh | |
| 140 | brew install zsh | |
| 141 | ``` | |
| 142 | ||
| 143 | ## Oh-My-Zsh | |
| 144 | ||
| 145 | I've shown the great aspects of [Oh My Zsh](https://ohmyz.sh) in other blog | |
| 146 | posts, so I'll skip over that speech for now. Simply install it and run an | |
| 147 | update. | |
| 148 | ||
| 149 | ```sh | |
| 150 | sh -c "$(curl -fsSL https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh)" | |
| 151 | ``` | |
| 152 | ||
| 153 | ```sh | |
| 154 | omz update | |
| 155 | ``` | |
| 156 | ||
| 157 | Finally, restart the iTerm2 application to ensure all changes go into effect. | |
| 158 | ||
| 159 | ## Oh-My-Zsh Themes | |
| 160 | ||
| 161 | Let's change the theme of the terminal to make it a little more friendly. | |
| 162 | ||
| 163 | ```sh | |
| 164 | open ~/.zshrc | |
| 165 | ``` | |
| 166 | ||
| 167 | The third section of this file should contain a line like the code below. Change | |
| 168 | that theme to | |
| 169 | [any theme you want](https://github.com/ohmyzsh/ohmyzsh/wiki/Themes), save the | |
| 170 | file, and exit. | |
| 171 | ||
| 172 | ```sh | |
| 173 | ZSH_THEME="af-magic" | |
| 174 | ``` | |
| 175 | ||
| 176 | After changing the `.zshrc` file, you'll need to close your terminal and re-open | |
| 177 | it to see the changes. | |
| 178 | Optionally, just open a new tab if you're using iTerm2, and you'll see the | |
| 179 | new shell config. | |
| 180 | ||
| 181 | ## Oh-My-Zsh Plugins | |
| 182 | ||
| 183 | Of course, my customization of `zsh` would not be complete without | |
| 184 | [zsh-autosuggestions](https://github.com/zsh-users/zsh-autosuggestions). This | |
| 185 | will bring up commands you've run in the past as you type them. For example, if | |
| 186 | you've run `ssh user@192.168.1.99` before, the terminal will show this command | |
| 187 | as soon as you start typing it (e.g. `zsh u`), and you can hit the right arrow | |
| 188 | to autocomplete the command. | |
| 189 | ||
| 190 | ```sh | |
| 191 | git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions | |
| 192 | ``` | |
| 193 | ||
| 194 | ```sh | |
| 195 | open ~/.zshrc | |
| 196 | ``` | |
| 197 | ||
| 198 | ```sh | |
| 199 | # Scroll down the script and edit this line to add zsh-autosuggestions | |
| 200 | plugins=(git zsh-autosuggestions) | |
| 201 | ``` | |
| 202 | ||
| 203 | Remember: After changing the `.zshrc` file, you'll need to close your terminal | |
| 204 | and re-open it to see the changes. Optionally, just open a new tab if you're | |
| 205 | using iTerm2, and you'll see the new shell config. | |
blog/2021-03-19-clone-github-repos.org added +153
| @@ -0,0 +1,153 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-03-19 | |
| 3 | title = "How to Clone All Repositories from a GitHub or Sourcehut Account" | |
| 4 | description = "Learn how to properly clone all repositories from a GitHub or sourcehut account." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Cloning from GitHub | |
| 9 | ||
| 10 | If you're like me and use a lot of different devices (and sometimes decide to | |
| 11 | just wipe your device and start with a new OS), you probably know the pain of | |
| 12 | cloning all your old code repositories down to your local file system. | |
| 13 | ||
| 14 | If you're using GitHub, you can easily clone all of your code back down in just | |
| 15 | seconds. | |
| 16 | First, create a bash script. | |
| 17 | I do so by opening a new file in `nano`, but you can use `gedit`, `vim`, or | |
| 18 | something else: | |
| 19 | ||
| 20 | ```sh | |
| 21 | nano clone_github_repos.sh | |
| 22 | ``` | |
| 23 | ||
| 24 | Next, paste in the following information. Note that you can replace the word | |
| 25 | `users` in the first line with `orgs` and type an organization's name instead of | |
| 26 | a user's name. | |
| 27 | ||
| 28 | ```sh | |
| 29 | CNTX=users; NAME=YOUR-USERNAME; PAGE=1 | |
| 30 | curl "https://api.github.com/$CNTX/$NAME/repos?page=$PAGE&per_page=100" | | |
| 31 | grep -e 'git_url*' | | |
| 32 | cut -d \" -f 4 | | |
| 33 | xargs -L1 git clone | |
| 34 | ``` | |
| 35 | ||
| 36 | Finally, save the bash script and make it executable. | |
| 37 | ||
| 38 | ```sh | |
| 39 | chmod a+x clone_github_repos.sh | |
| 40 | ``` | |
| 41 | ||
| 42 | Now you can run the script and should see the cloning process begin. | |
| 43 | ||
| 44 | ```sh | |
| 45 | ./clone_github_repos.sh | |
| 46 | ``` | |
| 47 | ||
| 48 | ## Cloning from Sourcehut | |
| 49 | ||
| 50 | I haven't fully figured out how to directly incorporate Sourcehut's GraphQL API | |
| 51 | into a bash script yet, so this one will take two steps. | |
| 52 | ||
| 53 | First, log-in to Sourcehut and go to their | |
| 54 | [GraphQL playground for Git](https://git.sr.ht/graphql). Next, paste the | |
| 55 | following query into the left box: | |
| 56 | ||
| 57 | ```sh | |
| 58 | query { | |
| 59 | me { | |
| 60 | canonicalName | |
| 61 | repositories() { | |
| 62 | cursor | |
| 63 | results { | |
| 64 | name | |
| 65 | } | |
| 66 | } | |
| 67 | } | |
| 68 | } | |
| 69 | ``` | |
| 70 | ||
| 71 | The output on the right side will give you an object of all your repositories. | |
| 72 | Just grab that text and remove all the characters such as quotation marks and | |
| 73 | curly brackets. You will need a single-line list of space-separated values for | |
| 74 | the next step. | |
| 75 | ||
| 76 | Now let's create the bash script: | |
| 77 | ||
| 78 | ```sh | |
| 79 | nano clone_sourcehut_repos.sh | |
| 80 | ``` | |
| 81 | ||
| 82 | Next, paste the following bash script in with the list of repositories you | |
| 83 | obtained above and replace `your-username` with your username. | |
| 84 | ||
| 85 | Note that this uses the SSH-based Git cloning method (e.g. `git@git...`), so | |
| 86 | you'll need to ensure you have set up Sourcehut with your SSH key. | |
| 87 | ||
| 88 | ```sh | |
| 89 | repos=(repo1 repo2 repo3) | |
| 90 | ||
| 91 | # List all sub-directories in the current directory | |
| 92 | for repo in "${repos[@]}" | |
| 93 | do | |
| 94 | # Clone | |
| 95 | git clone git@git.sr.ht:~your-username/$repo | |
| 96 | done | |
| 97 | ``` | |
| 98 | ||
| 99 | Finally, save the bash script and make it executable. | |
| 100 | ||
| 101 | ```sh | |
| 102 | chmod a+x clone_sourcehut_repos.sh | |
| 103 | ``` | |
| 104 | ||
| 105 | Now you can run the script and should see the cloning process begin. | |
| 106 | ||
| 107 | ```sh | |
| 108 | ./clone_sourcehut_repos.sh | |
| 109 | ``` | |
| 110 | ||
| 111 | ## Moving Repositories to a New Host | |
| 112 | ||
| 113 | Now that you have all of your code repositories cloned to your local computer, | |
| 114 | you may want to change the remote host (e.g., moving from GitHub to GitLab). | |
| 115 | To do this, let's create another bash script: | |
| 116 | ||
| 117 | ```sh | |
| 118 | nano change_remote_urls.sh | |
| 119 | ``` | |
| 120 | ||
| 121 | Past the following information and be sure to change the URL information to | |
| 122 | whichever host you are moving to. For this example, I am looping through all of | |
| 123 | my cloned GitHub directories and changing them to Sourcehut (e.g. | |
| 124 | `<YOUR_NEW_REMOTE_URL>` -\> `git@git.sr.ht:~myusername`). | |
| 125 | ||
| 126 | ```sh | |
| 127 | # List all sub-directories in the current directory | |
| 128 | for dir in */ | |
| 129 | do | |
| 130 | # Remove the trailing "/" | |
| 131 | dir=${dir%*/} | |
| 132 | # Enter sub-directory | |
| 133 | cd $dir | |
| 134 | # Change remote Git URL | |
| 135 | git remote set-url origin <YOUR_NEW_REMOTE_URL>/"${dir##*/}" | |
| 136 | # Push code to new remote | |
| 137 | git push | |
| 138 | # Go back to main directory | |
| 139 | cd .. | |
| 140 | done | |
| 141 | ``` | |
| 142 | ||
| 143 | Finally, save the bash script and make it executable. | |
| 144 | ||
| 145 | ```sh | |
| 146 | chmod a+x change_remote_urls.sh | |
| 147 | ``` | |
| 148 | ||
| 149 | Now you can run the script and should see the cloning process begin. | |
| 150 | ||
| 151 | ```sh | |
| 152 | ./change_remote_urls.sh | |
| 153 | ``` | |
blog/2021-03-28-gemini-capsule.org added +178
| @@ -0,0 +1,178 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-03-28 | |
| 3 | title = "Launching a Gemini Capsule" | |
| 4 | description = "Learn more about Gemini, an alternative internet protocol to the World Wide Web (WWW)" | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## What is Gemini? | |
| 9 | ||
| 10 | [Gemini](https://gemini.circumlunar.space/) is an internet protocol introduced | |
| 11 | in June 2019 as an alternative to HTTP(S) or Gopher. In layman's terms, it's an | |
| 12 | alternative way to browse sites (called capsules) that requires a special | |
| 13 | browser. Since Gemini is not standardized as an internet standard, normal web | |
| 14 | browsers won't be able to load a Gemini capsule. | |
| 15 | Instead, you'll need to use [a Gemini-specific browser](https://gemini. | |
| 16 | circumlunar.space/clients.html). | |
| 17 | ||
| 18 | The content found within a Gemini page is called | |
| 19 | [Gemtext](https://gemini.circumlunar.space/docs/cheatsheet.gmi) and is | |
| 20 | _extremely_ basic (on purpose). | |
| 21 | Gemini only processes the text, no media content like images. However, | |
| 22 | you're able to style 3 levels of headings, regular text, links (which will | |
| 23 | display on their own line), quotes, and an unordered list. | |
| 24 | ||
| 25 | Here's a complete listing of valid Gemtext: | |
| 26 | ||
| 27 | ```txt | |
| 28 | # Heading 1 | |
| 29 | ## Heading 2 | |
| 30 | ### Heading 3 | |
| 31 | ||
| 32 | Regular text! Lorem ipsum dolor sit amet. | |
| 33 | ||
| 34 | => https://example.com My Website | |
| 35 | => gemini://example.com My Gemini Capsule | |
| 36 | ||
| 37 | > "If life were predictable it would cease to be life, and be without flavor." - Eleanor Roosevelt | |
| 38 | ||
| 39 | My List: | |
| 40 | * Item | |
| 41 | * Item | |
| 42 | ||
| 43 | ```Anything between three backticks will be rendered as code.``` | |
| 44 | ``` | |
| 45 | ||
| 46 | ## Free Option | |
| 47 | ||
| 48 | There are probably numerous websites that allow you to create your personal | |
| 49 | Gemini capsule, but I'm going to focus on the two sites that I have personally | |
| 50 | tested. The first option below, Midnight Pub, allows you to create/edit any | |
| 51 | Gemini files you want in your account. This is essentially a GUI option with a | |
| 52 | built-in text box for editing. The second option below, Sourcehut, allows you to | |
| 53 | use a Git repository and automatic build process to deploy your personal Gemini | |
| 54 | capsule every time you push a commit. | |
| 55 | ||
| 56 | ### Midnight Pub - Beginner Friendly | |
| 57 | ||
| 58 | [Midnight Pub](https://midnight.pub/) is a small, virtual community meant to | |
| 59 | reflect the atmosphere of wandering into a small alley pub. The site is built in | |
| 60 | Gemtext and has a server-side process to convert Gemtext to HTML if someone | |
| 61 | loads the site in an HTTP(S) browser. | |
| 62 | ||
| 63 | To create an account, you'll need to email the owner of the website to obtain a | |
| 64 | key. | |
| 65 | You can find their email on the Midnight Pub homepage. | |
| 66 | Once registered, head to [your account](https://midnight.pub/account) and select | |
| 67 | [manage site](https://midnight.pub/site). This is the screen where you can | |
| 68 | upload or create any files to be displayed on the internet. | |
| 69 | ||
| 70 | For example, I've created both an HTML file and a Gemini file. Remember that | |
| 71 | Gemini is automatically converted to HTML on the Pub, so you don't need an HTML | |
| 72 | version. | |
| 73 | For example, I created an HTML version to add in some extra styling. | |
| 74 | ||
| 75 | All you need to do is create a page like `index.gmi` and use your Gemini browser | |
| 76 | to head over to your-username.midnight.pub to see the result. | |
| 77 | ||
| 78 | That's all there is to it! Easy enough, right? Let's check out a more advanced | |
| 79 | version in the next section. | |
| 80 | ||
| 81 | ## Paid Option | |
| 82 | ||
| 83 | As of 2021, Sourcehut has decided to require users to have a paid account in | |
| 84 | order to utilize their automated build system. For now, paid accounts can be as | |
| 85 | low as $2/month. | |
| 86 | ||
| 87 | ### Sourcehut | |
| 88 | ||
| 89 | [Sourcehut](https://sourcehut.org/) is a collection of software development | |
| 90 | tools, but mostly surrounds their hosted Git repository service. Simply put, | |
| 91 | it's a minimal and more private alternative to services like GitHub. | |
| 92 | ||
| 93 | This walkthrough is more advanced and involves things like Git, SSH, the command | |
| 94 | line. If you don't think you know enough to do this, check out my walkthrough on | |
| 95 | creating a Gemini capsule for the Midnight Pub instead. | |
| 96 | ||
| 97 | The first thing you'll need to do is create an SSH key pair, if you don't | |
| 98 | already have one on your system. Once created, grab the contents of `id_rsa.pub` | |
| 99 | and add it to your Sourcehut account settings - this will allow you to push and | |
| 100 | pull code changes without using a username/password. | |
| 101 | ||
| 102 | ```sh | |
| 103 | ssh keygen | |
| 104 | ``` | |
| 105 | ||
| 106 | Next up, let's create a repository with the proper name so that the Sourcehut | |
| 107 | build system will know we want them to host a website for us. Use the following | |
| 108 | format exactly: | |
| 109 | ||
| 110 | ```sh | |
| 111 | mkdir your-username.srht.site && cd your-username.srht.site | |
| 112 | ``` | |
| 113 | ||
| 114 | Now that we've created the repo, let's initialize Git and add the proper remote | |
| 115 | URL. | |
| 116 | ||
| 117 | ```sh | |
| 118 | git init | |
| 119 | ``` | |
| 120 | ||
| 121 | ```sh | |
| 122 | git remote add origin git@git.sr.ht:~your-username/your-username.srht.site | |
| 123 | ``` | |
| 124 | ||
| 125 | Now that our repository is set up and configured, we will need to create at | |
| 126 | least two files: | |
| 127 | ||
| 128 | - `index.gmi` | |
| 129 | - `.build.yml` | |
| 130 | ||
| 131 | For your `.build.yml` file, use the following content and be sure to update the | |
| 132 | `site` line with your username! | |
| 133 | ||
| 134 | ```yaml | |
| 135 | image: alpine/latest | |
| 136 | oauth: pages.sr.ht/PAGES:RW | |
| 137 | environment: | |
| 138 | site: your-username.srht.site | |
| 139 | tasks: | |
| 140 | - package: | | |
| 141 | cd $site | |
| 142 | tar -cvz . > ../site.tar.gz | |
| 143 | - upload: | | |
| 144 | acurl -f https://pages.sr.ht/publish/$site -Fcontent=@site.tar.gz -Fprotocol=GEMINI | |
| 145 | ``` | |
| 146 | ||
| 147 | For the `index.gmi` file, put whatever you want in there and save it. You could | |
| 148 | even just copy and paste the Gemtext cheatsheet. | |
| 149 | ||
| 150 | If you want to serve both HTML and Gemini files from this repository, just add a | |
| 151 | second command to the `upload` section: | |
| 152 | ||
| 153 | ```yaml | |
| 154 | - upload: | | |
| 155 | acurl -f https://pages.sr.ht/publish/$site -Fcontent=@site.tar.gz -Fprotocol=GEMINI | |
| 156 | acurl -f https://pages.sr.ht/publish/$site -Fcontent=@site.tar.gz | |
| 157 | ``` | |
| 158 | ||
| 159 | Lastly, commit your changes and push them to the remote repo. | |
| 160 | ||
| 161 | ```sh | |
| 162 | git add .; git commit -m "initial commit"; git push --set-upstream origin HEAD | |
| 163 | ``` | |
| 164 | ||
| 165 | If you've successfully created the files with the proper format, you'll see the | |
| 166 | terminal print a message that lets you know where the automatic build is taking | |
| 167 | place. For example, here's what the terminal tells me: | |
| 168 | ||
| 169 | ```sh | |
| 170 | remote: Build started: | |
| 171 | remote: https://builds.sr.ht/~user/job/689803 [.build.yml] | |
| 172 | ``` | |
| 173 | ||
| 174 | Now that you've properly built your Sourcehut page, you can browse to | |
| 175 | your-username.srht.site in a Gemini browser and view the final results. Take a | |
| 176 | look at the image below for my Sourcehut Gemini capsule. | |
| 177 | ||
| 178 |  | |
blog/2021-03-28-vaporwave-vs-outrun.org added +120
| @@ -0,0 +1,120 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-03-28 | |
| 3 | title = "Vaporwave vs Outrun" | |
| 4 | description = "This post discusses Vaporwave and Outrun, which have become almost synonymous in the minds of those online who aren't hardcore into these genres of media." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | When it comes to an aesthetic that originated primarily online, there tends to | |
| 10 | be a lot of confusion around what falls into each specific genre. This post | |
| 11 | discusses Vaporwave and Outrun, which have become almost synonymous in the minds | |
| 12 | of those online who aren't hardcore into these genres of media. More | |
| 13 | specifically, Outrun is largely an unknown term while all aesthetics in these | |
| 14 | two genres are usually attributed to Vaporwave. For example, take a look at the | |
| 15 | image search results for Vaporwave: the results include **a lot** of | |
| 16 | Outrun-themed images. You'll find a similar trend almost everywhere. | |
| 17 | ||
| 18 |  | |
| 19 | ||
| 20 | ## Vaporwave | |
| 21 | ||
| 22 |  | |
| 23 | ||
| 24 | Okay, so what is Vaporwave? I'm going to just copy-and-paste some general info | |
| 25 | from the Wikipedia article on | |
| 26 | [Vaporwave](https://en.wikipedia.org/wiki/Vaporwave), so that I'm not repeating | |
| 27 | everything you can already search for online: | |
| 28 | ||
| 29 | > Vaporwave is a microgenre of electronic music, a visual art style, and an | |
| 30 | > Internet meme that emerged in the early 2010s. It is defined partly by its | |
| 31 | > slowed-down, chopped and screwed samples of smooth jazz, elevator, R&B, and | |
| 32 | > lounge music from the 1980s and 1990s. The surrounding subculture is sometimes | |
| 33 | > associated with an ambiguous or satirical take on consumer capitalism and pop | |
| 34 | > culture, and tends to be characterized by a nostalgic or surrealist engagement | |
| 35 | > with the popular entertainment, technology and advertising of previous | |
| 36 | > decades. Visually, it incorporates early Internet imagery, late 1990s web | |
| 37 | > design, glitch art, anime, 3D-rendered objects, and cyberpunk tropes in its | |
| 38 | > cover artwork and music videos. | |
| 39 | ||
| 40 | This is an excellent summary, and it helps address my point here: there are | |
| 41 | specific aspects that make Vaporwave unique: | |
| 42 | ||
| 43 | ### Time Frame | |
| 44 | ||
| 45 | The time frame for references, logos, etc. focuses mostly on the 1990s in | |
| 46 | Vaporwave. You'll see old school Pepsi logos, Microsoft 95 screens, tropical | |
| 47 | plants, classic marble sculptures, and many references from Japan's influence in | |
| 48 | the 90s. | |
| 49 | ||
| 50 | ### Art | |
| 51 | ||
| 52 | The color scheme is generally a soft, light palette that uses pastel colors. The | |
| 53 | backdrop will often be in a light-pink or teal blue. | |
| 54 | ||
| 55 | ### Music | |
| 56 | ||
| 57 | The musical genre of Vaporwave incorporates soft tunes, lounge music, and sound | |
| 58 | effects that will make the listener reminisce of the 90s. The sounds of the | |
| 59 | genre are generally slower-paced and calming. The major breakthrough artist for | |
| 60 | Vaporwave was [Macintosh Plus](https://en.wikipedia.org/wiki/Vektroid), who | |
| 61 | released the album | |
| 62 | [Floral Shoppe](https://archive.org/details/MACINTOSHPLUS-FLORALSHOPPE_complete) | |
| 63 | in 2011. Another more recent example is the artist | |
| 64 | [sadbert](https://sadbert.bandcamp.com/), whose latest album incorporates the | |
| 65 | upbeat tones of the 1999 Dilbert TV series. | |
| 66 | ||
| 67 | Notice that Vaporwave doesn't include things like racing cars, futuristic | |
| 68 | technology, chrome, or the deep orange/purple color scheme. Vaporwave is a focus | |
| 69 | on the idyllic state of the world when technology was becoming common in | |
| 70 | households, a reality that we have already experienced. Focus on the most | |
| 71 | aesthetically-pleasing parts of that past is a large part of Vaporwave. | |
| 72 | ||
| 73 | ## Outrun | |
| 74 | ||
| 75 |  | |
| 76 | ||
| 77 | Now, let's get to Outrun. This one is a little trickier since the genre has | |
| 78 | largely been lumped under the Vaporwave title for so long. However, it stems | |
| 79 | from the [Synthwave](https://en.wikipedia.org/wiki/Synthwave) music genre and is | |
| 80 | likely named after the 1986 racer game, | |
| 81 | [Out Run](https://en.wikipedia.org/wiki/Out_Run). | |
| 82 | ||
| 83 | ### Time Frame | |
| 84 | ||
| 85 | Outrun can be thought of as a retro-futuristic aesthetic born from the 1980s. | |
| 86 | ||
| 87 | ### Art | |
| 88 | ||
| 89 | The color scheme uses a very dark color palette with the main colors being deep | |
| 90 | oranges, blues, and purples. Red edges are common around objects in Outrun art. | |
| 91 | The background of the Outrun aesthetic is almost always a neon grid like you'd | |
| 92 | expect to see in Tron or a 1980s arcade machine. | |
| 93 | ||
| 94 | Classic sports cars, chrome robots, computer generated graphics and fonts, and | |
| 95 | the occasional use of rain or palm trees can be found in Outrun art. | |
| 96 | ||
| 97 | ### Music | |
| 98 | ||
| 99 | This aesthetic has a more aggressive and fast-paced style of music, which tends | |
| 100 | to match the subject of the art in this aesthetic. | |
| 101 | ||
| 102 | Outrun enthusiasts love what people in the 1980s thought the future would look | |
| 103 | like. Take a look at a common video game discussed in Outrun circles, Far Cry 3: | |
| 104 | Blood Dragon: | |
| 105 | ||
| 106 |  | |
| 107 | ||
| 108 | Another example that doesn't force the color scheme as hard as some online art | |
| 109 | does is Kung Fury: | |
| 110 | ||
| 111 |  | |
| 112 | ||
| 113 |  | |
| 114 | ||
| 115 | ## Conclusion | |
| 116 | ||
| 117 | While Vaporwave and Outrun share similarities, they are two distinct aesthetics | |
| 118 | with many important distinctions. Someone who enjoys one may not necessarily | |
| 119 | enjoy the other, so it's important to make sure we properly describe the | |
| 120 | aesthetic we're looking for. | |
blog/2021-03-30-vps-web-server.org added +400
| @@ -0,0 +1,400 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-03-30 | |
| 3 | title = "How to Set Up a VPS Web Server" | |
| 4 | description = "Choosing a place to host a website is one of the most confusing decisions for beginner web developers. Even for experienced web devs, choosing between different forms of web hosting can be a daunting choice." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Shared Hosting vs. VPS | |
| 9 | ||
| 10 | Choosing a place to host a website is one of the most confusing decisions for | |
| 11 | beginner web developers. Even for experienced web devs, choosing between | |
| 12 | different forms of web hosting can be a daunting choice. | |
| 13 | ||
| 14 | First, let's take a look at | |
| 15 | [shared web hosting](https://en.wikipedia.org/wiki/Shared_web_hosting_service). | |
| 16 | Shared web hosting is a product where you are purchasing a small piece of a web | |
| 17 | server that is being shared between many websites. As a result, the cost of | |
| 18 | shared hosting is extremely low. You won't have access to the server itself, | |
| 19 | which means you can't install your own software on the server, such as Docker. | |
| 20 | Usually, you are simply allowed to connect your domains to the server, set up | |
| 21 | domain security, and other small utilities. | |
| 22 | ||
| 23 | In contrast, a | |
| 24 | [virtual private server](https://en.wikipedia.org/wiki/Virtual_private_server) | |
| 25 | (VPS) is a virtual machine that replicates the environment of having a dedicated | |
| 26 | server to yourself. You are able to control the virtual server's host names, | |
| 27 | base file system, package manager, etc. Another great upside of a VPS is that | |
| 28 | since it's virtual, the company providing the VPS can dynamically increase the | |
| 29 | disk size, RAM size, or number of CPUs at any time. However, the virtual server | |
| 30 | is still physically located on a server that is shared between multiple virtual | |
| 31 | servers. | |
| 32 | ||
| 33 | The choice between shared hosting and VPS mostly depends on your skill level | |
| 34 | with system administration. If you're comforting working on a server that is | |
| 35 | mostly left up to you (or you're willing to learn), then a VPS is usually a | |
| 36 | better option. However, shared hosting is a fantastic option for people who | |
| 37 | don't want to have to learn how to manage their server. | |
| 38 | ||
| 39 | ## My Situation | |
| 40 | ||
| 41 | I had used shared hosting for approximately 5 years before trying my first VPS. | |
| 42 | I manage a homelab and have had success running a server and performing typical | |
| 43 | sysadmin duties, but I was still hesitant to get a VPS. One fear was that I | |
| 44 | always struggled to properly set up the networking part of a server - DNS and | |
| 45 | hostname configurations were not my friend. | |
| 46 | ||
| 47 | As a little bit of background, I originally used | |
| 48 | [Siteground](https://www.siteground.com) for my initially shared hosting and | |
| 49 | stayed on that platform for at least a year. However, the UI was clunky, and I | |
| 50 | didn't like how they handled certain technical aspects, so I switched to | |
| 51 | [Namecheap](https://www.namecheap.com). Namecheap was great because it is the | |
| 52 | service I primarily use for purchasing domain names, which made it incredibly | |
| 53 | easy to link them to my hosting service. However, it was still mediocre shared | |
| 54 | hosting, and Namecheap is notorious for not letting you use | |
| 55 | [Let's Encrypt](https://letsencrypt.org) to obtain free SSL/TLS certificates; | |
| 56 | Namecheap wants to make you purchase certificates through their store. | |
| 57 | ||
| 58 | Finally, I settled down with [iWebFusion](https://www.iwebfusion.net) for about | |
| 59 | the last year of my shared hosting. This service was pretty great, came with | |
| 60 | free SSL/TLS, and I never had any complaints. | |
| 61 | ||
| 62 | However, I finally grew tired of not being able to install software on my own | |
| 63 | web server. I wanted to be able to try out things like | |
| 64 | [Postmill](https://postmill.xyz) or [Matrix](https://matrix.org). This is | |
| 65 | possible with a VPS, so I decided to grab a new domain name to try it out. | |
| 66 | ||
| 67 | ## Getting Started: Buying a VPS | |
| 68 | ||
| 69 | The first step to moving over to a VPS is (you guessed it): finding a VPS | |
| 70 | provider. For my VPSs, I use [1984](https://1984hosting.com) and prefer their | |
| 71 | services much more than any alternative, due to their location (Iceland), their | |
| 72 | [privacy policy](https://1984hosting.com/GDPR/), their respect for GDPR, and the | |
| 73 | ability to remain anonymous if you pay in Bitcoin or Monero. | |
| 74 | ||
| 75 | [Njalla](https://njal.la) is another good, privacy-oriented option for VPS | |
| 76 | services. | |
| 77 | ||
| 78 | You'll have to decide what specifications you want on your VPS. For me, I only | |
| 79 | build and deploy low-resource HTML, PHP, and Python websites. This means I can | |
| 80 | survive on the smallest VPS: 1 CPU, 1GB of RAM, and 25GB SSD for $5.00 per | |
| 81 | month. | |
| 82 | ||
| 83 | As noted above, the great thing about a VPS is you can request your provider to | |
| 84 | increase the resources at any time. | |
| 85 | ||
| 86 | ## Configuring DNS Settings | |
| 87 | ||
| 88 | Okay, so now let's get into some actual work that has to be done to get content | |
| 89 | moved from a shared host to a VPS. At this point, I'm assuming you have a shared | |
| 90 | host with website content that you can still access, and you've purchased a new | |
| 91 | VPS and can SSH into that server. | |
| 92 | ||
| 93 | The first change is minor, but it should be done immediately in order to | |
| 94 | get things moving: DNS settings. Go to wherever your DNS settings are handled. | |
| 95 | If your shared host also managed your DNS settings, you'll need to first move | |
| 96 | that DNS over to your new VPS provider. For me, I route my DNS through | |
| 97 | [Gandi](https://www.gandi.net). | |
| 98 | ||
| 99 | Once you know where your DNS settings are, go ahead and update the `A` records | |
| 100 | to match the public IP address of your VPS. For example: | |
| 101 | ||
| 102 | ```txt | |
| 103 | A example.com xxx.xxx.xxx.xxx | |
| 104 | A subdomain xxx.xxx.xxx.xxx | |
| 105 | CNAME www example.com. | |
| 106 | ``` | |
| 107 | ||
| 108 | If you have any other records that require updates, such as MX or TXT records | |
| 109 | for a mail server, be sure to update those accordingly. Personally, I don't host | |
| 110 | my own mail server. | |
| 111 | I route all mail on my custom domains to [Migadu](https://www.migadu.com). | |
| 112 | Hosting your own email server can become complex quickly and is not for | |
| 113 | beginners. | |
| 114 | ||
| 115 | DNS changes can take up to 48 hours to propagate, so be sure to give it some | |
| 116 | time before assuming you've made an error. | |
| 117 | ||
| 118 | ## Server Updates and Packages | |
| 119 | ||
| 120 | Now that the DNS settings have been changed, let's set up our server while we | |
| 121 | wait for the DNS to propagate. | |
| 122 | First up is to ssh into your server. | |
| 123 | If you've signed up with a service like DigitalOcean, you can add your SSH | |
| 124 | key to your account and to your VPS droplet so that you don't need a | |
| 125 | password in order to SSH. | |
| 126 | ||
| 127 | ```sh | |
| 128 | ssh root@xxx.xxx.xxx.xxx | |
| 129 | ``` | |
| 130 | ||
| 131 | The VPS that is used in this blog post runs Ubuntu 20.04 with an Apache web | |
| 132 | server. | |
| 133 | If you're working on a different operating system (OS) or want a | |
| 134 | different web server, such as Nginx, you'll have to use different commands | |
| 135 | to set it up. | |
| 136 | ||
| 137 | First, let's update and upgrade our server. | |
| 138 | ||
| 139 | **NOTE:** Since we have logged in to the server as `root` for now, we don't need | |
| 140 | to use the `sudo` modifier before our commands. | |
| 141 | ||
| 142 | ```sh | |
| 143 | apt update && apt upgrade -y | |
| 144 | ``` | |
| 145 | ||
| 146 | ## Create A User Account | |
| 147 | ||
| 148 | While being able to use `root` can be beneficial at times, you shouldn't use | |
| 149 | `root` unless you have to. | |
| 150 | ||
| 151 | So let's set up a new user in our system. The `-m` option below tells the OS to | |
| 152 | create a home directory for the new user. | |
| 153 | ||
| 154 | ```sh | |
| 155 | adduser USERNAME | |
| 156 | ``` | |
| 157 | ||
| 158 | Now, create a password for that user. | |
| 159 | ||
| 160 | ```sh | |
| 161 | passwd USERNAME | |
| 162 | ``` | |
| 163 | ||
| 164 | Finally, add the user to the sudoers file, so they can perform priveleged | |
| 165 | commands. | |
| 166 | ||
| 167 | ```sh | |
| 168 | usermod -a -G sudo USERNAME | |
| 169 | ``` | |
| 170 | ||
| 171 | If you are using SSH keys and not passwords, you'll need to copy your SSH key | |
| 172 | from your local machine to the VPS. If you haven't disabled password-based SSH | |
| 173 | yet, the easiest way to do this is `ssh-copy-id` from your local computer (not | |
| 174 | from the VPS): | |
| 175 | ||
| 176 | ```sh | |
| 177 | ssh-copy-id testuser@xxx.xxx.xxx.xxx | |
| 178 | ``` | |
| 179 | ||
| 180 | If you've disabled password-based SSH, you'll need to manually copy your SSH key | |
| 181 | into the `~/.ssh/authorized_keys` file. | |
| 182 | ||
| 183 | ## Install Software | |
| 184 | ||
| 185 | Our goal here is to host a web server, so the next step is to install the Apache | |
| 186 | web server and any other packages we need. | |
| 187 | ||
| 188 | From this point on, I will be logged in as a user (not `root`) and will need to | |
| 189 | use the `sudo` modifier for most commands. | |
| 190 | ||
| 191 | ```sh | |
| 192 | sudo apt update; sudo apt upgrade -y; sudo apt autoremove -y | |
| 193 | sudo apt install apache2 | |
| 194 | ``` | |
| 195 | ||
| 196 | If you need other language support, such as PHP, you'll need to install that | |
| 197 | too. | |
| 198 | ||
| 199 | ```sh | |
| 200 | sudo apt install libapache2-mod-php php-dom | |
| 201 | sudo a2enmod php | |
| 202 | sudo systemctl restart apache2 | |
| 203 | ``` | |
| 204 | ||
| 205 | ## Website Files & Folders | |
| 206 | ||
| 207 | Next up is to create the directories for the domain(s) we want to be hosted on | |
| 208 | this web server. | |
| 209 | ||
| 210 | ```sh | |
| 211 | cd /var/www | |
| 212 | sudo mkdir example.com | |
| 213 | ``` | |
| 214 | ||
| 215 | We have a folder for `example.com` now, so let's add an `index.html` file and | |
| 216 | put it within a specific `public_html` folder. You don't need this `public_html` | |
| 217 | if you don't want it, but it helps with organizing items related to | |
| 218 | `example.com` that you don't want to publish to the internet. | |
| 219 | ||
| 220 | ```sh | |
| 221 | cd example.com | |
| 222 | sudo mkdir public_html && cd public_html | |
| 223 | sudo nano index.html | |
| 224 | ``` | |
| 225 | ||
| 226 | You can put anything you want in this `index.html` file. If you can't think of | |
| 227 | anything, paste this in there: | |
| 228 | ||
| 229 | ```html | |
| 230 | <!DOCTYPE html> | |
| 231 | <html lang="en"> | |
| 232 | <head> | |
| 233 | <meta charset="utf-8" /> | |
| 234 | <meta name="viewport" content="width=device-width, initial-scale=1" /> | |
| 235 | <title>Hello, world!</title> | |
| 236 | </head> | |
| 237 | <body> | |
| 238 | <h1>Hello, world!</h1> | |
| 239 | </body> | |
| 240 | </html> | |
| 241 | ``` | |
| 242 | ||
| 243 | If you want something to be served at `example.com/page01/file.txt`, you'll have | |
| 244 | to create the `page01` directory under the `example.com` directory. For example: | |
| 245 | ||
| 246 | ```sh | |
| 247 | cd /var/www/example.com/public_html | |
| 248 | sudo mkdir page01 | |
| 249 | sudo nano file.txt | |
| 250 | ``` | |
| 251 | ||
| 252 | ## Apache Configuration | |
| 253 | ||
| 254 | Now, let's set up the files that will tell the server where to find the files | |
| 255 | for `example.com`. We will copy the default configuration file and create our | |
| 256 | own. | |
| 257 | ||
| 258 | ```sh | |
| 259 | cd /etc/apache2/sites-available | |
| 260 | sudo cp 000-default.conf example.com.conf | |
| 261 | sudo nano example.com.conf | |
| 262 | ``` | |
| 263 | ||
| 264 | This configuration file will have a few default lines, but you'll need to edit | |
| 265 | it to look similar to this (settings may change based on your personal needs): | |
| 266 | ||
| 267 | ```config | |
| 268 | <VirtualHost *:80> | |
| 269 | ServerAdmin your-email@email-provider.com | |
| 270 | ServerName example.com | |
| 271 | ServerAlias www.example.com | |
| 272 | DocumentRoot /var/www/example.com/public_html | |
| 273 | ErrorLog ${APACHE_LOG_DIR}/error.log | |
| 274 | CustomLog ${APACHE_LOG_DIR}/access.log combined | |
| 275 | </VirtualHost> | |
| 276 | ``` | |
| 277 | ||
| 278 | Now, enable the configuration for your new site, disable the default | |
| 279 | configuration, and reload the web server. | |
| 280 | ||
| 281 | ```sh | |
| 282 | sudo a2ensite example.com.conf | |
| 283 | sudo a2dissite 000-default.conf | |
| 284 | sudo systemctl reload apache2 | |
| 285 | ``` | |
| 286 | ||
| 287 | You can always run a test to make sure no errors or warnings are found in your | |
| 288 | configuration files. | |
| 289 | ||
| 290 | ```sh | |
| 291 | sudo apache2ctl configtest | |
| 292 | ``` | |
| 293 | ||
| 294 | Now, restart the web server entirely. After this, you should be able to browse | |
| 295 | to `http://example.com` and see the HTML content you provided earlier. Note that | |
| 296 | SSL/TLS has not been enabled yet, so you won't be able to use the secure version | |
| 297 | yet (`https://example.com`). | |
| 298 | ||
| 299 | ```sh | |
| 300 | sudo systemctl restart apache2 | |
| 301 | ``` | |
| 302 | ||
| 303 | You can repeat this for as many websites as you need. Just create the domain | |
| 304 | folders in `/var/www/`, add the configuration file, enable the configuration, | |
| 305 | and restart `apache2`. | |
| 306 | ||
| 307 | ## SSL/TLS Certificates: Serve Websites Over HTTPS | |
| 308 | ||
| 309 | In order to serve secure content, you'll need to obtain SSL/TLS certificates. | |
| 310 | Luckily, there's a free tool called [Certbot](https://certbot.eff.org) that | |
| 311 | helps us with the process. | |
| 312 | ||
| 313 | The first step is to install `snapd` and `core` for Ubuntu. | |
| 314 | ||
| 315 | ```sh | |
| 316 | sudo apt install snapd | |
| 317 | sudo snap install core | |
| 318 | sudo snap refresh core | |
| 319 | ``` | |
| 320 | ||
| 321 | Next, install the `certbot` snap package. | |
| 322 | ||
| 323 | ```sh | |
| 324 | sudo snap install --classic certbot | |
| 325 | ``` | |
| 326 | ||
| 327 | Execute the following command to ensure that the `certbot` command can be run. | |
| 328 | ||
| 329 | ```sh | |
| 330 | sudo ln -s /snap/bin/certbot /usr/bin/certbot | |
| 331 | ``` | |
| 332 | ||
| 333 | Finally, you can run `certbot` one of two ways: | |
| 334 | ||
| 335 | 1. run it and let it alter your Apache configuration files automatically to | |
| 336 | enable HTTPS redirects. | |
| 337 | 2. run it and only allow it to create certificates. | |
| 338 | You'll need to manually alter the config files to enable HTTPS redirects. | |
| 339 | ||
| 340 | Run certbot and allow automatic config changes: | |
| 341 | ||
| 342 | ```sh | |
| 343 | sudo certbot --apache | |
| 344 | ``` | |
| 345 | ||
| 346 | Run certbot for certificates only and don't allow it to alter config files: | |
| 347 | ||
| 348 | ```sh | |
| 349 | sudo certbot certonly --apache | |
| 350 | ``` | |
| 351 | ||
| 352 | The Certbot packages on your system come with a cron job or systemd timer that | |
| 353 | will renew your certificates automatically before they expire. You will not need | |
| 354 | to run Certbot again unless you change your configuration. You can test | |
| 355 | automatic renewal for your certificates by running this command: | |
| 356 | ||
| 357 | ```sh | |
| 358 | sudo certbot renew --dry-run | |
| 359 | ``` | |
| 360 | ||
| 361 | Now, test your domains by going to `https://example.com`. | |
| 362 | ||
| 363 | ## Firewall Security | |
| 364 | ||
| 365 | To enable better security on your server, you'll need to enable a basic | |
| 366 | firewall. For Ubuntu, we'll use | |
| 367 | [the uncomplicated firewall](https://0x4b1d.org/blog/secure-your-network-with-the-uncomplicated-firewall.html). | |
| 368 | ||
| 369 | Now, add the following rules to the firewall allow SSH, Apache, and HTTP(S) | |
| 370 | connections. If you need to, you can enable different ports for specifics | |
| 371 | applications, SFTP, etc. | |
| 372 | ||
| 373 | ```sh | |
| 374 | sudo ufw default deny incoming | |
| 375 | sudo ufw default allow outgoing | |
| 376 | sudo ufw allow OpenSSH | |
| 377 | sudo ufw allow Apache | |
| 378 | sudo ufw allow proto tcp from any to any port 80,443 | |
| 379 | ``` | |
| 380 | ||
| 381 | Once you've added all the rules you need, enable the firewall. | |
| 382 | ||
| 383 | ```sh | |
| 384 | sudo ufw enable | |
| 385 | ``` | |
| 386 | ||
| 387 | ## Troubleshooting | |
| 388 | ||
| 389 | If you run into any issues during your VPS set-up, be sure to walk back through | |
| 390 | your actions and make sure you didn't miss any steps. | |
| 391 | ||
| 392 | Many websites have fantastic guides to setting up various web servers. | |
| 393 | This is one of the areas [where DigitalOcean shines](https://www.digitalocean. | |
| 394 | com/community/tutorials). | |
| 395 | For simpler or more Linux-oriented questions, I suggest using [Linuxize] | |
| 396 | (https://linuxize.com). | |
| 397 | ||
| 398 | If you're getting certain errors (e.g. `500 Internal Server Error`) and need to | |
| 399 | debug locally, you can view the `access.log` and `error.log` files in the | |
| 400 | `/var/log/apache/` directory. | |
blog/2021-04-17-gemini-server.org added +159
| @@ -0,0 +1,159 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-04-17 | |
| 3 | title = "Hosting a Gemini Server" | |
| 4 | description = "Dive deeper into Gemini by hosting your own Gemini web server." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Similar Article Available | |
| 9 | ||
| 10 | To read more about Gemini and ways to test out this new protocol without your | |
| 11 | own server, see my previous post | |
| 12 | [Launching a Gemini Capsule](/blog/launching-a-gemini-capsule/). | |
| 13 | ||
| 14 | ## Preparation | |
| 15 | ||
| 16 | This guide assumes you have access to a server accessible to the world through a | |
| 17 | public IP address and that you own a domain name used for this Gemini capsule. | |
| 18 | ||
| 19 | ## Getting Started with Agate | |
| 20 | ||
| 21 | We are going to use [Agate](https://github.com/mbrubeck/agate) for this | |
| 22 | tutorial. | |
| 23 | This is a basic Gemini server written in Rust. | |
| 24 | It takes very little time and maintenance to get it running. | |
| 25 | ||
| 26 | ## Install Dependencies | |
| 27 | ||
| 28 | First, you will need to install the Rust package for your system. On Ubuntu, use | |
| 29 | the following commands (remember to use `sudo` if you are not the root user). | |
| 30 | The Rust installation will give you options to customize the installation; I | |
| 31 | used the default installation options. | |
| 32 | ||
| 33 | ```sh | |
| 34 | sudo apt update && sudo apt upgrade -y | |
| 35 | curl https://sh.rustup.rs -sSf | sh | |
| 36 | ``` | |
| 37 | ||
| 38 | Remember to configure your shell with the new configuration: | |
| 39 | ||
| 40 | ```sh | |
| 41 | source $HOME/.cargo/env | |
| 42 | ``` | |
| 43 | ||
| 44 | Before we install agate, make sure you have the `gcc` package installed: | |
| 45 | ||
| 46 | ```sh | |
| 47 | sudo apt install gcc | |
| 48 | ``` | |
| 49 | ||
| 50 | Next, you'll need to install the agate executable with Rust's Cargo package | |
| 51 | maintainer: | |
| 52 | ||
| 53 | ```sh | |
| 54 | cargo install agate | |
| 55 | ``` | |
| 56 | ||
| 57 | ## Create Symlinks | |
| 58 | ||
| 59 | Once Cargo has finished installing all the required packages, symlink the | |
| 60 | executable to your \$PATH. | |
| 61 | ||
| 62 | ```sh | |
| 63 | sudo ln -s $HOME/.cargo/bin/agate /usr/local/bin/agate | |
| 64 | ``` | |
| 65 | ||
| 66 | ## Using Agate's Built-In Installation Tool | |
| 67 | ||
| 68 | If you're running Ubuntu or Debian, use the Debian installation script found in | |
| 69 | Agate's GitHub repository, under the `tools/debian` folder. | |
| 70 | ||
| 71 | ```sh | |
| 72 | git clone https://github.com/mbrubeck/agate | |
| 73 | cd agate/tools/debian | |
| 74 | sudo ./install.sh | |
| 75 | ``` | |
| 76 | ||
| 77 | ## Configure the Gemini Service | |
| 78 | ||
| 79 | We have a little more to do, but since this script tries to immediately run the | |
| 80 | service, it will likely fail with an exit code. Let's add our finishing touches. | |
| 81 | Edit the following file and replace the hostname with your desired URL. You can | |
| 82 | also change the directory where content will be served. | |
| 83 | ||
| 84 | ```sh | |
| 85 | sudo nano /etc/systemd/system/gemini.service | |
| 86 | ``` | |
| 87 | ||
| 88 | ```sh | |
| 89 | # Edit these lines to whatever you want - see the next code block for my personal configuration. | |
| 90 | WorkingDirectory=/srv/gemini | |
| 91 | ExecStart=agate --hostname $(uname -n) --lang en | |
| 92 | ``` | |
| 93 | ||
| 94 | This is my personal config: | |
| 95 | ||
| 96 | ```sh | |
| 97 | WorkingDirectory=/var/gemini/ | |
| 98 | ExecStart=agate --hostname gemini.example.com --lang en | |
| 99 | ``` | |
| 100 | ||
| 101 | Since we've altered the systemd configuration files, we have to reload the | |
| 102 | daemon. Let's do that, restart our service, and check its status. | |
| 103 | ||
| 104 | ```sh | |
| 105 | sudo systemctl daemon-reload | |
| 106 | sudo systemctl restart gemini.service | |
| 107 | sudo systemctl status gemini.service | |
| 108 | ``` | |
| 109 | ||
| 110 | ## Fixing Systemd Errors | |
| 111 | ||
| 112 | If you're still getting errors, the installation process may not have properly | |
| 113 | enabled the gemini service. Fix it with the following commands. | |
| 114 | ||
| 115 | ```sh | |
| 116 | sudo systemctl enable gemini.service | |
| 117 | sudo systemctl restart gemini.service | |
| 118 | sudo systemctl status gemini.service | |
| 119 | ``` | |
| 120 | ||
| 121 | ## Firewall Rules | |
| 122 | ||
| 123 | Great! Our server is now functional and running. The first consideration now is | |
| 124 | that you need to be able to access port 1965 on the server. If you have a | |
| 125 | firewall enabled, you'll need to open that port up. | |
| 126 | ||
| 127 | ```sh | |
| 128 | sudo ufw allow 1965 | |
| 129 | sudo ufw reload | |
| 130 | ``` | |
| 131 | ||
| 132 | ## Creating Content | |
| 133 | ||
| 134 | Let's create the Gemini capsule. Note that wherever you set the WorkingDirectory | |
| 135 | variable to earlier, Agate will expect you to put your Gemini capsule contents | |
| 136 | in a sub-folder called "content." So, I place my files in "/var/gmi/content." I'm | |
| 137 | going to create that folder now and put a file in there. | |
| 138 | ||
| 139 | ```sh | |
| 140 | sudo mkdir /var/gemini/content | |
| 141 | sudo nano /var/gemini/content/index.gmi | |
| 142 | ``` | |
| 143 | ||
| 144 | You can put whatever you want in the "index.gmi" file, just make sure it's valid | |
| 145 | Gemtext. | |
| 146 | ||
| 147 | ## The Results | |
| 148 | ||
| 149 | Here are some screenshots of the Gemini page I just created in the | |
| 150 | [Lagrange](https://gmi.skyjake.fi/lagrange/) browser and the | |
| 151 | [amfora](https://github.com/makeworld-the-better-one/amfora) browser. | |
| 152 | ||
| 153 |  | |
| 154 | ||
| 155 | _Lagrange_ | |
| 156 | ||
| 157 |  | |
| 158 | ||
| 159 | _Amfora_ | |
blog/2021-04-23-php-comment-system.org added +262
| @@ -0,0 +1,262 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-04-23 | |
| 3 | title = "Roll Your Own Static Commenting System in PHP" | |
| 4 | description = "Learn how to skip the tedium of finding the proper commenting system by simply rolling your own in PHP." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## The Terrible-ness of Commenting Systems | |
| 9 | ||
| 10 | The current state of affairs regarding interactive comment systems is, well, | |
| 11 | terrible. It is especially awful if you're a privacy conscious person who does | |
| 12 | not generally load third-party scripts or frames on the websites you visit. | |
| 13 | ||
| 14 | Even further, many comment systems are charging exorbitant fees for something | |
| 15 | that should be standard. | |
| 16 | ||
| 17 | Of course, there are some really terrible options: | |
| 18 | ||
| 19 | - Facebook Comments | |
| 20 | - Discourse | |
| 21 | ||
| 22 | There are some options that are better but still use too many scripts, frames, | |
| 23 | or social integrations on your web page that could impact some users: | |
| 24 | ||
| 25 | - Disqus | |
| 26 | - Isso | |
| 27 | - Remark42 | |
| 28 | ||
| 29 | Lastly, I looked into a few unique ways of generating blog comments, such as | |
| 30 | using Twitter threads or GitHub issues to automatically post issues. However, | |
| 31 | these both rely on external third-party sites that I don't currently use. | |
| 32 | ||
| 33 | ## Stay Static with Server-Side Comments | |
| 34 | ||
| 35 | The main issue for my personal use-case is that my blog is completely, 100% | |
| 36 | static. I use PHP on the back-end but website visitors only see HTML and a | |
| 37 | single CSS file. No external javascript and no embedded frames. | |
| 38 | ||
| 39 | So, how do we keep a site static and still allow users to interact with blog | |
| 40 | posts? The key actually pretty simple - I'm already using PHP, so why not rely | |
| 41 | on the classic HTML `<form>` and a PHP script to save the comments somewhere? As | |
| 42 | it turns out, this was a perfect solution for me. | |
| 43 | ||
| 44 | The second issue for my personal use-case is that I am trying to keep the | |
| 45 | contents of my website accessible over time, as described by @brandur, in his | |
| 46 | post entitled | |
| 47 | [Blog with Markdown + Git, and degrade gracefully through time](https://brandur.org/fragments/graceful-degradation-time) | |
| 48 | . | |
| 49 | ||
| 50 | This means I cannot rely on a database for comments, since I do not rely on a | |
| 51 | database for any other part of my websites. | |
| 52 | ||
| 53 | I blog in plain Markdown files, commit all articles to Git, and ensure that | |
| 54 | future readers will be able to see the source data long after I'm gone, or the | |
| 55 | website has gone offline. However, I still haven't committed any images served | |
| 56 | on my blog to Git, as I'm not entirely sold on Git LFS yet - for now, images can | |
| 57 | be found at [img.0x4b1d.org](https://img.0x4b1d.org). | |
| 58 | ||
| 59 | Saving my comments back to the Git repository ensures that another aspect of my | |
| 60 | site will degrade gracefully. | |
| 61 | ||
| 62 | ## Create a Comment Form | |
| 63 | ||
| 64 | Okay, let's get started. The first step is to create an HTML form that users can | |
| 65 | see and utilize to submit comments. This is fairly easy and can be changed | |
| 66 | depending on your personal preferences. | |
| 67 | ||
| 68 | Take a look at the code block below for the form I currently use. Note that | |
| 69 | `<current-url>` is replaced automatically in PHP with the current post's URL, so | |
| 70 | that my PHP script used later will know which blog post the comment is related | |
| 71 | to. | |
| 72 | ||
| 73 | The form contains the following structure: | |
| 74 | ||
| 75 | 1. `<form>` - This is the form and will determine which PHP script to send the | |
| 76 | comment to. | |
| 77 | 2. `<section hidden>` - This section is hidden from the user and is used to | |
| 78 | ensure that we know which blog post sent the comment. | |
| 79 | 3. `<section>` Display Name (Optional) - Used to accept a display name, if | |
| 80 | entered. | |
| 81 | 4. `<section>` Comment (Required) - Used to accept the user's full comment. | |
| 82 | Markdown is allowed. | |
| 83 | 5. `<button>` - A button to submit the form. | |
| 84 | ||
| 85 | ```html | |
| 86 | <form action="/comment.php" method="POST"> | |
| 87 | <h3>Leave a Comment</h3> | |
| 88 | <section hidden> | |
| 89 | <label class="form-label" for="postURL">Post URL</label> | |
| 90 | <input | |
| 91 | class="form-control" | |
| 92 | id="postURL" | |
| 93 | name="postURL" | |
| 94 | type="text" | |
| 95 | value="<current-url>" | |
| 96 | /> | |
| 97 | </section> | |
| 98 | <section> | |
| 99 | <label class="form-label" for="userName">Display Name</label> | |
| 100 | <input | |
| 101 | class="form-control" | |
| 102 | id="userName" | |
| 103 | name="userName" | |
| 104 | placeholder="John Doe" | |
| 105 | type="text" | |
| 106 | /> | |
| 107 | </section> | |
| 108 | <section> | |
| 109 | <label class="form-label" for="userContent">Your Comment</label> | |
| 110 | <textarea | |
| 111 | class="form-control" | |
| 112 | id="userContent" | |
| 113 | name="userContent" | |
| 114 | rows="3" | |
| 115 | placeholder="# Feel free to use Markdown" | |
| 116 | aria-describedby="commentHelp" | |
| 117 | required | |
| 118 | ></textarea> | |
| 119 | <div id="commentHelp" class="form-text"> | |
| 120 | Comments are saved as Markdown and cannot be edited or deleted. | |
| 121 | </div> | |
| 122 | </section> | |
| 123 | <button type="submit">Submit</button> | |
| 124 | </form> | |
| 125 | ``` | |
| 126 | ||
| 127 | ## Handle Comments via POST | |
| 128 | ||
| 129 | Now that we have a form and can submit comments, we need to create a PHP script | |
| 130 | so that the server can fetch the comment data and save it. Make sure your PHP | |
| 131 | script name matches the name you entered in the `action` field in your form. | |
| 132 | ||
| 133 | See the code block below for the contents of my `comment.php` script. We perform | |
| 134 | the following tasks in this script: | |
| 135 | ||
| 136 | 1. Grab the POST data from the HTML form. | |
| 137 | 2. Sanitize the comment data with `htmlentities`. | |
| 138 | 3. Set the display name to `Anonymous` if it was left blank. | |
| 139 | 4. Create a PHP object that combines all of this data. | |
| 140 | 5. Check if our destination file `comments.json` exists. | |
| 141 | 6. If so, convert the PHP object to JSON and save it to the file. | |
| 142 | 7. If the `comments.json` file does not exist, the script will exit with an | |
| 143 | error. You can alter this to ensure it creates the script, but my source code | |
| 144 | includes the file by default, so it will always exist. | |
| 145 | 8. Finally, send the user back to the `#comments` section of the blog post they | |
| 146 | just read. | |
| 147 | ||
| 148 | ```php | |
| 149 | // Get the content sent from the comment form | |
| 150 | $comment = htmlentities($_POST['userContent']); | |
| 151 | $post_url = $_POST['postURL']; | |
| 152 | ||
| 153 | // Set default values if blank | |
| 154 | if (isset($_POST['userName']) && trim($_POST['userName']) !== "") { | |
| 155 | $username = $_POST['userName']; | |
| 156 | } else { | |
| 157 | $username = 'Anonymous'; | |
| 158 | } | |
| 159 | ||
| 160 | // Create an empty PHP object | |
| 161 | $user_object = new stdClass(); | |
| 162 | ||
| 163 | // Add object content | |
| 164 | $user_object->timestamp = date('Y-m-d H:i:s'); | |
| 165 | $user_object->username = $username; | |
| 166 | $user_object->comment = $comment; | |
| 167 | $user_object->post_url = $post_url; | |
| 168 | ||
| 169 | // Append JSON to file | |
| 170 | $file_name = 'comments.json'; | |
| 171 | if (file_exists($file_name)) { | |
| 172 | $source_data = file_get_contents($file_name); | |
| 173 | $temp_array = json_decode($source_data); | |
| 174 | array_push($temp_array, $user_object); | |
| 175 | $json_data = json_encode($temp_array, JSON_PRETTY_PRINT); | |
| 176 | file_put_contents($file_name, $json_data); | |
| 177 | } else { | |
| 178 | die('Error: The "comments.json" file does not exist.'); | |
| 179 | } | |
| 180 | ||
| 181 | // Send the user back | |
| 182 | header('Location: ' . $post_url . '#comments'); | |
| 183 | ``` | |
| 184 | ||
| 185 | If you're using Apache, make sure the `www-data` user on your server has the | |
| 186 | correct permissions to your website directory or else it will not be able to | |
| 187 | write to `comments.json`. | |
| 188 | ||
| 189 | ```sh | |
| 190 | chgrp -R www-data /path/to/website/ | |
| 191 | chmod -R g+w comments.json | |
| 192 | ``` | |
| 193 | ||
| 194 | ## Display User Comments | |
| 195 | ||
| 196 | Now that we can submit comments, and they are saved to the `comments.json` file, | |
| 197 | let's make sure we can show those comments in each blog post. | |
| 198 | ||
| 199 | The code block below shows the function I use to decode my `comments.json` file, | |
| 200 | check if the comments apply to the current post, and then display them. | |
| 201 | ||
| 202 | This piece of code should **really** be inside a function (or at least in an | |
| 203 | organized PHP workflow). Don't just copy-and-paste and expect it to work. You | |
| 204 | need to at least supply a `$query` variable depending on the page visited. | |
| 205 | ||
| 206 | ```php | |
| 207 | $query = 'your-blog-post.html'; | |
| 208 | ||
| 209 | // Load saved comments | |
| 210 | $comments_file = 'comments.json'; | |
| 211 | $comments_raw = file_get_contents($comments_file); | |
| 212 | $comments = json_decode($comments_raw); | |
| 213 | $comment_section = '<section id="comments" class="comments"><h3>Comments</h3>'; | |
| 214 | foreach ($comments as $comment) { | |
| 215 | if ($comment->post_url == "https://example.com/post/" . $query) { | |
| 216 | // Assign metadata to variables | |
| 217 | $comment_timestamp = $comment->timestamp; | |
| 218 | $comment_username = $comment->username; | |
| 219 | $comment_content = $comment->comment; | |
| 220 | ||
| 221 | // Parse the comment, in case it contains Markdown | |
| 222 | $comment_md = Parsedown::instance()->text($comment_content); | |
| 223 | $comment_html = new DOMDocument(); | |
| 224 | $comment_html->loadHTML($comment_md); | |
| 225 | $comment_html_links = $comment_html->getElementsByTagName('a'); | |
| 226 | foreach ($comment_html_links as $comment_html_link) { | |
| 227 | $comment_html_link->setAttribute('rel', 'noreferrer'); | |
| 228 | $comment_html_link->setAttribute('target', '_blank'); | |
| 229 | } | |
| 230 | $comment_secured_html = $comment_html->saveHTML(); | |
| 231 | ||
| 232 | // Apply metadata to comments section | |
| 233 | $comment_section .= '<div class="user-comment"><div class="row"><label>Timestamp:</label><p>' . $comment_timestamp . '</p></div><div class="row"><label>Name:</label><p>' . $comment_username . '</p></div><div class="row markdown"><label>Comment:</label><div class="comment-markdown">' . $comment_secured_html . '</div></div></div>'; | |
| 234 | } | |
| 235 | } | |
| 236 | ||
| 237 | echo $comment_section; | |
| 238 | ``` | |
| 239 | ||
| 240 | ## Bonus: Create a 'Recent Comments' Page | |
| 241 | ||
| 242 | Finally, the last part of my current system is to create a Recent Comments page | |
| 243 | so that I can easily check-in on my blog and moderate any spam. As an | |
| 244 | alternative, you could use PHP's `mail()` function to email you for each blog | |
| 245 | comment. | |
| 246 | ||
| 247 | The code to do this is literally the same code as the previous section, I just | |
| 248 | make sure it is printed when someone visits `https://example.com/comments/`. | |
| 249 | ||
| 250 | ## Possible Enhancements | |
| 251 | ||
| 252 | This comment system is by no means a fully-developed system. I have noted a few | |
| 253 | possible enhancements here that I may implement in the future: | |
| 254 | ||
| 255 | - Create a secure moderator page with user authentication at | |
| 256 | `https://blog.example.com/mod/`. This page could have the option to edit or | |
| 257 | delete any comment found in `comments.json`. | |
| 258 | - Create a temporary file, such as `pending_comments.json`, that will store | |
| 259 | newly-submitted comments and won't display on blog posts until approved by a | |
| 260 | moderator. | |
| 261 | - Create a `/modlog/` page with a chronological log, showing which moderator | |
| 262 | approved which comments and why certain comments were rejected. | |
blog/2021-04-28-photography.org added +67
| @@ -0,0 +1,67 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-04-28 | |
| 3 | title = "Jumping Back Into Photography" | |
| 4 | description = "After a long respite, I have returned to my hobby of photography with a brand-new camera." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Why Photography? | |
| 9 | ||
| 10 | I've often wondered why photography is as enticing as it is. You can see | |
| 11 | billions of people around the world taking photographs every single moment of | |
| 12 | the day. New technology often boasts about their photographic capabilities, | |
| 13 | especially smartphones. I would even assume that we live in a world where there | |
| 14 | is never a moment in which a photograph is not being taken somewhere on Earth. | |
| 15 | ||
| 16 | As for myself, I would simply say that I enjoy preserving a memory in physical | |
| 17 | (or digital) form. I've never had the best memory when it comes to recalling | |
| 18 | details of places and people gone by, so it helps to have a frame of reference | |
| 19 | lying around. | |
| 20 | ||
| 21 | Regardless of the reason, I think most people would agree that you simply cannot | |
| 22 | have too many hobbies. | |
| 23 | ||
| 24 | ## Older Cameras | |
| 25 | ||
| 26 | I started playing around with the idea of photography when my family purchased a | |
| 27 | Fujifilm camera for family-specific events. I don't recall the specific model, | |
| 28 | but I do recall it was a point-and-shoot camera without an interchangeable lens. | |
| 29 | However, it was of great value to someone, like myself, who couldn't afford any | |
| 30 | other camera. I took about 10,000 shots with that camera over a 3-5 year span. | |
| 31 | Most notably, all of my trips to California were documented through this camera. | |
| 32 | ||
| 33 | When possible, I would borrow my sister's camera, which is a Sony SLT-A58. This | |
| 34 | camera was great and allowed for some of my best early shots, especially those | |
| 35 | taken in Utah's and Nevada's parks. | |
| 36 | ||
| 37 | ## My Current Kit | |
| 38 | ||
| 39 | I've finally come to a point in my life where I have the disposable income to | |
| 40 | invest in a solid photography kit. I played around with the idea of a lot of | |
| 41 | different cameras, different types, new vs used, etc. Finally, I settled on the | |
| 42 | [Sony α7 III](https://en.wikipedia.org/wiki/Sony_%CE%B17_III). This camera is | |
| 43 | mirror-less and uses a full-frame image sensor at 24 megapixels. I don't create | |
| 44 | large prints, and I am mostly focused on preserving memories in high quality for | |
| 45 | the next 5-10 years with this camera, so the specifications here are just | |
| 46 | perfect for me. | |
| 47 | ||
| 48 | For lenses, I decided to buy two lenses that could carry me through most | |
| 49 | situations: | |
| 50 | ||
| 51 | - [Vario-Tessar T\* FE 24-70 mm F4 ZA OSS](https://electronics.sony.com/imaging/lenses/full-frame-e-mount/p/sel2470z) | |
| 52 | - [Tamron 70-300mm f4.5-6.3 Di III RXD](https://www.tamron-usa.com/product/lenses/a047.html) | |
| 53 | ||
| 54 | In addition, I grabbed a couple | |
| 55 | [HGX Prime 67mm](https://www.promaster.com/Product/6725) protection filters for | |
| 56 | the lenses. | |
| 57 | ||
| 58 | As I delve further into photography and pick up more skills, I will most likely | |
| 59 | go back and grab a lens with a higher f-stop value, such as f/1.8. I toyed with | |
| 60 | the idea of grabbing a 50 mm at `f/1.8`, but decided to keep things in a | |
| 61 | reasonable price range instead. | |
| 62 | ||
| 63 | Finally, I made sure to buy a photography-specific backpack with a rain guard, | |
| 64 | and the zipper on the back panel, to protect the equipment while wearing the | |
| 65 | bag. If you've ever had to haul around a DSLR (or camera of similar heft) in a | |
| 66 | bag that only has a shoulder strap, you'll know the pain it can cause. Putting | |
| 67 | all my equipment in a backpack was an easy decision. | |
blog/2021-05-30-changing-git-authors.org added +75
| @@ -0,0 +1,75 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-05-30 | |
| 3 | title = "Changing Git Authors" | |
| 4 | description = "Learn how to haphazardly replace Git authors and/or emails within the Git history." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Changing Git Author/Email Based on Previously Committed Email | |
| 8 | ||
| 9 | Here's the dilemma: You've been committing changes to your git repository with | |
| 10 | an incorrect name or email (or multiple repositories), and now you want to fix | |
| 11 | it. | |
| 12 | Luckily, there's a semi-reliable way to fix that. | |
| 13 | While I have never experienced issues with this method, some people have | |
| 14 | warned that it can mess with historical hashes and integrity of commits, so | |
| 15 | use this method only if you're okay accepting that risk. | |
| 16 | ||
| 17 | Okay, let's create the bash script: | |
| 18 | ||
| 19 | ```sh | |
| 20 | nano change_git_authors.sh | |
| 21 | ``` | |
| 22 | ||
| 23 | The following information can be pasted directly into your bash script. The only | |
| 24 | changes you need to make are to the following variables: | |
| 25 | ||
| 26 | - `OLD_EMAIL` | |
| 27 | - `CORRECT_NAME` | |
| 28 | - `CORRECT_EMAIL` | |
| 29 | ||
| 30 | ```sh | |
| 31 | #!/bin/sh | |
| 32 | ||
| 33 | # List all sub-directories in the current directory | |
| 34 | for dir in */ | |
| 35 | do | |
| 36 | # Remove the trailing "/" | |
| 37 | dir=${dir%*/} | |
| 38 | # Enter sub-directory | |
| 39 | cd $dir | |
| 40 | ||
| 41 | git filter-branch --env-filter ' | |
| 42 | ||
| 43 | OLD_EMAIL="old@example.com" | |
| 44 | CORRECT_NAME="your-new-name" | |
| 45 | CORRECT_EMAIL="new@example.com" | |
| 46 | ||
| 47 | if [ "$GIT_COMMITTER_EMAIL" = "$OLD_EMAIL" ] | |
| 48 | then | |
| 49 | export GIT_COMMITTER_NAME="$CORRECT_NAME" | |
| 50 | export GIT_COMMITTER_EMAIL="$CORRECT_EMAIL" | |
| 51 | fi | |
| 52 | if [ "$GIT_AUTHOR_EMAIL" = "$OLD_EMAIL" ] | |
| 53 | then | |
| 54 | export GIT_AUTHOR_NAME="$CORRECT_NAME" | |
| 55 | export GIT_AUTHOR_EMAIL="$CORRECT_EMAIL" | |
| 56 | fi | |
| 57 | ' --tag-name-filter cat -- --branches --tags | |
| 58 | ||
| 59 | git push --force --tags origin 'refs/heads/*' | |
| 60 | ||
| 61 | cd .. | |
| 62 | done | |
| 63 | ``` | |
| 64 | ||
| 65 | Finally, save the bash script and make it executable. | |
| 66 | ||
| 67 | ```sh | |
| 68 | chmod a+x change_git_authors.sh | |
| 69 | ``` | |
| 70 | ||
| 71 | Now you can run the script and should see the process begin. | |
| 72 | ||
| 73 | ```sh | |
| 74 | ./change_git_authors.sh | |
| 75 | ``` | |
blog/2021-07-15-delete-gitlab-repos.org added +115
| @@ -0,0 +1,115 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-07-15 | |
| 3 | title = "How to Delete All GitLab Repositories" | |
| 4 | description = "Learn how to delete all GitLab repositories in your user account." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Background | |
| 9 | ||
| 10 | Have you ever used GitLab to host your source code, moved to a different host, | |
| 11 | and wanted to delete everything from your GitLab account? | |
| 12 | Well, this post covers any scenario where you would want to delete all | |
| 13 | repositories from your GitLab account. | |
| 14 | ||
| 15 | For me, I currently maintain around 30 repositories and don't like to manually | |
| 16 | delete them whenever I switch host. | |
| 17 | GitHub has a few different tools online to delete all repositories for you, | |
| 18 | but I have not found anything similar for GitLab, so I needed an alternative | |
| 19 | solution. | |
| 20 | ||
| 21 | ## Use a Python Script | |
| 22 | ||
| 23 | ### Requirements | |
| 24 | ||
| 25 | Before we look at the script, make sure you know your GitLab username. Next, | |
| 26 | [create an authorization token](https://gitlab.com/-/profile/personal_access_tokens) | |
| 27 | so that the Python script can delete your repositories. Don't lose this token or | |
| 28 | else you'll need to create a new one. | |
| 29 | ||
| 30 | ### Create the Script | |
| 31 | ||
| 32 | To run a Python script, you must first create it. Open a terminal and enter the | |
| 33 | following commands in whichever directory you prefer to store the script. You | |
| 34 | can do the same things in a file manager if you prefer. | |
| 35 | ||
| 36 | ```sh | |
| 37 | mkdir delete-gitlab | |
| 38 | ``` | |
| 39 | ||
| 40 | ```sh | |
| 41 | cd delete-gitlab | |
| 42 | ``` | |
| 43 | ||
| 44 | ```sh | |
| 45 | nano main.py | |
| 46 | ``` | |
| 47 | ||
| 48 | Enter the following code into your `main.py` script. | |
| 49 | ||
| 50 | ```python | |
| 51 | import request | |
| 52 | import json | |
| 53 | ||
| 54 | ||
| 55 | def get_project_ids(): | |
| 56 | url = "https://gitlab.com/api/v4/users/{user-id}/projects" | |
| 57 | ||
| 58 | querystring = {"owned": "true", "simple": "true", "per_page": "50"} | |
| 59 | ||
| 60 | payload = "" | |
| 61 | headers = {'authorization': 'Bearer {auth-token}'} | |
| 62 | ||
| 63 | response = requests.request("GET", url, data=payload, headers=headers, params=querystring) | |
| 64 | ||
| 65 | projects = json.loads(response.text) | |
| 66 | projects_ids = list(map(lambda project: project.get('id'), projects)) | |
| 67 | ||
| 68 | return projects_ids | |
| 69 | ||
| 70 | ||
| 71 | def remove_project(project_id): | |
| 72 | url_temp = "https://gitlab.com/api/v4/projects/{project}" | |
| 73 | headers = {'authorization': 'Bearer {auth-token}'} | |
| 74 | querystring = "" | |
| 75 | payload = "" | |
| 76 | ||
| 77 | url = url_temp.format(project=project_id) | |
| 78 | ||
| 79 | response = requests.request("DELETE", url, data=payload, headers=headers, params=querystring) | |
| 80 | project = json.loads(response.text) | |
| 81 | print(project) | |
| 82 | ||
| 83 | ||
| 84 | def main(): | |
| 85 | projects_ids = get_project_ids() | |
| 86 | ||
| 87 | url_temp = "https://gitlab.com/api/v4/projects/{project}" | |
| 88 | headers = {'authorization': 'Bearer {auth-token}'} | |
| 89 | querystring = "" | |
| 90 | payload = "" | |
| 91 | ||
| 92 | for project_id in projects_ids: | |
| 93 | url = url_temp.format(project=project_id) | |
| 94 | ||
| 95 | response = requests.request("GET", url, data=payload, headers=headers, params=querystring) | |
| 96 | project = json.loads(response.text) | |
| 97 | print(str(project.get('id')) + " " + project.get('name')) | |
| 98 | print("Removing...") | |
| 99 | remove_project(project_id) | |
| 100 | ||
| 101 | ||
| 102 | if __name__ == "__main__": | |
| 103 | main() | |
| 104 | ``` | |
| 105 | ||
| 106 | Now that you have the proper information, replace `{user-id}` with your GitLab | |
| 107 | username and `{auth-token}` with the authorization token you created earlier. | |
| 108 | ||
| 109 | Finally, simply run the script and watch the output. You can also use PyCharm | |
| 110 | Community Edition to edit and run the Python script if you don't want to work | |
| 111 | in a terminal. | |
| 112 | ||
| 113 | ```sh | |
| 114 | python3 main.py | |
| 115 | ``` | |
blog/2021-08-25-audit-sampling.org added +277
| @@ -0,0 +1,277 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-08-25 | |
| 3 | title = "Audit Sampling with Python" | |
| 4 | description = "Learn how to use Python to automate the boring parts of audit sampling." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Introduction | |
| 9 | ||
| 10 | For anyone who is familiar with internal auditing, external auditing, or | |
| 11 | consulting, you will understand how tedious audit testing can become when you | |
| 12 | are required to test large swaths of data. When we cannot establish an automated | |
| 13 | means of testing an entire population, we generate samples to represent the | |
| 14 | population of data. This helps ensure we can have a small enough data pool to | |
| 15 | test and that our results still represent the population. | |
| 16 | ||
| 17 | However, sampling data within the world of audit still seems to confuse quite a | |
| 18 | lot of people. While some audit-focused tools have introduced sampling | |
| 19 | functionality (e.g. Wdesk), many audit departments and firms cannot use software | |
| 20 | like this due to certain constraints, such as the team's budget or knowledge. | |
| 21 | Here is where this article comes in: we're going to use | |
| 22 | [Python](https://www.python.org), a free and open-source programming language, | |
| 23 | to generate random samples from a dataset in order to suffice numerous audit | |
| 24 | situations. | |
| 25 | ||
| 26 | ## Audit Requirements for Sampling | |
| 27 | ||
| 28 | Before we get into the details of how to sample with Python, I want to make sure | |
| 29 | I discuss the different requirements that auditors may have of samples used | |
| 30 | within their projects. | |
| 31 | ||
| 32 | ### Randomness | |
| 33 | ||
| 34 | First, let's discuss randomness. When testing out new technology to help assist | |
| 35 | with audit sampling, you need to understand exactly how your samples are being | |
| 36 | generated. For example, if the underlying function is just picking every 57th | |
| 37 | element from a list, that's not truly random; it's a systematic form of | |
| 38 | sampling. Luckily, since Python is open-source, we have access to its codebase. | |
| 39 | Through this blog post, I will be using the [pandas](https://pandas.pydata.org) | |
| 40 | module in order to generate the random samples. More specifically, I will be | |
| 41 | using the | |
| 42 | [pandas.DataFrame.sample](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.sample.html) | |
| 43 | function provided by Pandas. | |
| 44 | ||
| 45 | Now that you know what you're using, you can always check out the code behind | |
| 46 | `pandas.DataFrame.sample`. This function does a lot of work, but we really only | |
| 47 | care about the following snippets of code: | |
| 48 | ||
| 49 | ```python | |
| 50 | # Process random_state argument | |
| 51 | rs = com.random_state(random_state) | |
| 52 | ||
| 53 | ... | |
| 54 | ||
| 55 | locs = rs.choice(axis_length, size=n, replace=replace, p=weights) | |
| 56 | result = self.take(locs, axis=axis) | |
| 57 | if ignore_index: | |
| 58 | result.index = ibase.default_index(len(result)) | |
| 59 | ||
| 60 | return result | |
| 61 | ``` | |
| 62 | ||
| 63 | The block of code above shows you that if you assign a `random_state` | |
| 64 | argument when you run the function, that will be used as a seed number in | |
| 65 | the random generation and will allow you to reproduce a sample, given that | |
| 66 | nothing else changes. | |
| 67 | This is critical to the posterity of audit work. | |
| 68 | After all, how can you say your audit process is adequately documented if | |
| 69 | the next person can't run the code and get the same sample? | |
| 70 | The final piece here on randomness is to look at the [choice](https://docs. | |
| 71 | python.org/3/library/random.html#random.choice) function used above. | |
| 72 | This is the crux of the generation and can also be examined for more | |
| 73 | detailed analysis on its reliability. | |
| 74 | As far as auditing goes, we will trust that these functions are | |
| 75 | mathematically random. | |
| 76 | ||
| 77 | ### Sample Sizes | |
| 78 | ||
| 79 | As mentioned in the intro, sampling is only an effective method of auditing | |
| 80 | when it truly represents the entire population. | |
| 81 | While some audit departments or firms may consider certain judgmental sample | |
| 82 | sizes to be adequate, you may need to rely on statistically-significant | |
| 83 | confidence levels of sample testing at certain points. | |
| 84 | I will demonstrate both here. | |
| 85 | For statistically-significant confidence levels, most people will assume a | |
| 86 | 90% - 99% confidence level. | |
| 87 | In order to actually calculate the correct sample size, it is best to use | |
| 88 | statistical tools due to the tedious math work required. | |
| 89 | For example, for a population of 1000, and a 90% confidence level that no | |
| 90 | more than 5% of the items are nonconforming, you would sample 45 items. | |
| 91 | ||
| 92 | However, in my personal experience, many audit departments and firms do not use | |
| 93 | statistical sampling. Most people use a predetermined, often proprietary, table | |
| 94 | that will instruct auditors which sample sizes to choose. This allows for | |
| 95 | uniform testing and reduces overall workload. See the table below for a common | |
| 96 | implementation of sample sizes: | |
| 97 | ||
| 98 | | Control Frequency | Sample Size - High Risk | Sample Size - Low Risk | | |
| 99 | |-------------------|-------------------------|------------------------| | |
| 100 | | More Than Daily | 40 | 25 | | |
| 101 | | Daily | 40 | 25 | | |
| 102 | | Weekly | 12 | 5 | | |
| 103 | | Monthly | 5 | 3 | | |
| 104 | | Quarterly | 2 | 2 | | |
| 105 | | Semi-Annually | 1 | 1 | | |
| 106 | | Annually | 1 | 1 | | |
| 107 | | Ad-hoc | 1 | 1 | | |
| 108 | ||
| 109 | ## Sampling with Python & Pandas | |
| 110 | ||
| 111 | In this section, I am going to cover a few basic audit situations that require | |
| 112 | sampling. While some situations may require more effort, the syntax, | |
| 113 | organization, and intellect used remain largely the same. If you've never used | |
| 114 | Python before, note that lines starting with a '`#`' symbol are called comments, | |
| 115 | and they will be skipped by Python. I highly recommend taking a quick tutorial | |
| 116 | online to understand the basics of Python if any of the code below is confusing | |
| 117 | to you. | |
| 118 | ||
| 119 | ### Simple Random Sample | |
| 120 | ||
| 121 | First, let's look at a simple, random sample. The code block below will import | |
| 122 | the `pandas` module, load a data file, sample the data, and export the sample to | |
| 123 | a file. | |
| 124 | ||
| 125 | ```python | |
| 126 | # Import the Pandas module | |
| 127 | import pandas | |
| 128 | ||
| 129 | # Specify where to find the input file & where to save the final sample | |
| 130 | file_input = r'Population Data.xlsx' | |
| 131 | file_output = r'Sample.xlsx' | |
| 132 | ||
| 133 | # Load the data with pandas | |
| 134 | # Remember to use the sheet_name parameter if your Excel file has multiple sheets | |
| 135 | df = pandas.read_excel(file_input) | |
| 136 | ||
| 137 | # Sample the data for 25 selections | |
| 138 | # Remember to always use the random_state parameter so the sample can be re-performed | |
| 139 | sample = df.sample(n=25, random_state=0) | |
| 140 | ||
| 141 | # Save the sample to Excel | |
| 142 | sample.to_excel(file_output) | |
| 143 | ``` | |
| 144 | ||
| 145 | ### Simple Random Sample: Using Multiple Input Files | |
| 146 | ||
| 147 | Now that we've created a simple sample, let's create a sample from multiple | |
| 148 | files. | |
| 149 | ||
| 150 | ```python | |
| 151 | # Import the Pandas module | |
| 152 | import pandas | |
| 153 | ||
| 154 | # Specify where to find the input file & where to save the final sample | |
| 155 | file_input_01 = r'Population Data Q1.xlsx' | |
| 156 | file_input_02 = r'Population Data Q2.xlsx' | |
| 157 | file_input_03 = r'Population Data Q3.xlsx' | |
| 158 | file_output = r'Sample.xlsx' | |
| 159 | ||
| 160 | # Load the data with pandas | |
| 161 | # Remember to use the sheet_name parameter if your Excel file has multiple sheets | |
| 162 | df_01 = pandas.read_excel(file_input_01) | |
| 163 | df_02 = pandas.read_excel(file_input_02) | |
| 164 | df_03 = pandas.read_excel(file_input_03) | |
| 165 | ||
| 166 | # Sample the data for 5 selections from each quarter | |
| 167 | # Remember to always use the random_state parameter so the sample can be re-performed | |
| 168 | sample_01 = df_01.sample(n=5, random_state=0) | |
| 169 | sample_02 = df_02.sample(n=5, random_state=0) | |
| 170 | sample_03 = df_03.sample(n=5, random_state=0) | |
| 171 | ||
| 172 | # If required, combine the samples back together | |
| 173 | sample = pandas.concat([sample_01, sample_02, sample_03], ignore_index=True) | |
| 174 | ||
| 175 | # Save the sample to Excel | |
| 176 | sample.to_excel(file_output) | |
| 177 | ``` | |
| 178 | ||
| 179 | ### Stratified Random Sample | |
| 180 | ||
| 181 | Well, what if you need to sample distinct parts of a single file? For example, | |
| 182 | let's write some code to separate our data by "Region" and sample those regions | |
| 183 | independently. | |
| 184 | ||
| 185 | ```python | |
| 186 | # Import the Pandas module | |
| 187 | import pandas | |
| 188 | ||
| 189 | # Specify where to find the input file & where to save the final sample | |
| 190 | file_input = r'Sales Data.xlsx' | |
| 191 | file_output = r'Sample.xlsx' | |
| 192 | ||
| 193 | # Load the data with pandas | |
| 194 | # Remember to use the sheet_name parameter if your Excel file has multiple sheets | |
| 195 | df = pandas.read_excel(file_input) | |
| 196 | ||
| 197 | # Stratify the data by "Region" | |
| 198 | df_east = df[df['Region'] == 'East'] | |
| 199 | df_west = df[df['Region'] == 'West'] | |
| 200 | ||
| 201 | # Sample the data for 5 selections from each quarter | |
| 202 | # Remember to always use the random_state parameter so the sample can be re-performed | |
| 203 | sample_east = df_east.sample(n=5, random_state=0) | |
| 204 | sample_west = df_west.sample(n=5, random_state=0) | |
| 205 | ||
| 206 | # If required, combine the samples back together | |
| 207 | sample = pandas.concat([sample_east, sample_west], ignore_index=True) | |
| 208 | ||
| 209 | # Save the sample to Excel | |
| 210 | sample.to_excel(file_output) | |
| 211 | ``` | |
| 212 | ||
| 213 | ### Stratified Systematic Sample | |
| 214 | ||
| 215 | This next example is quite useful if you need audit coverage over a certain time | |
| 216 | period. This code will generate samples for each month in the data and combine | |
| 217 | them all together at the end. Obviously, this code can be modified to stratify | |
| 218 | by something other than months, if needed. | |
| 219 | ||
| 220 | ```python | |
| 221 | # Import the Pandas module | |
| 222 | import pandas | |
| 223 | ||
| 224 | # Specify where to find the input file & where to save the final sample | |
| 225 | file_input = r'Sales Data.xlsx' | |
| 226 | file_output = r'Sample.xlsx' | |
| 227 | ||
| 228 | # Load the data with pandas | |
| 229 | # Remember to use the sheet_name parameter if your Excel file has multiple sheets | |
| 230 | df = pandas.read_excel(file_input) | |
| 231 | ||
| 232 | # Convert the date column to datetime so the function below will work | |
| 233 | df['Date of Sale'] = pandas.to_datetime(df['Date of Sale']) | |
| 234 | ||
| 235 | # Define a function to create a sample for each month | |
| 236 | def monthly_stratified_sample(df: pandas.DataFrame, date_column: str, num_selections: int) -> pandas.DataFrame: | |
| 237 | static_num_selections = num_selections final_sample = pandas.DataFrame() | |
| 238 | for month in range(1, 13): | |
| 239 | num_selections = static_num_selections | |
| 240 | rows_list = [] | |
| 241 | for index, row in df.iterrows(): | |
| 242 | df_month = row[date_column].month | |
| 243 | if month == df_month: | |
| 244 | rows_list.append() | |
| 245 | monthly_df = pd.DataFrame(data=rows_list) | |
| 246 | if (len(monthly_df)) == 0: | |
| 247 | continue | |
| 248 | elif not (len(monthly_df) > sample_size): | |
| 249 | num_selections = sample_size | |
| 250 | elif len(monthly_df) >= sample_size: | |
| 251 | num_selections = sample_size | |
| 252 | sample = monthly_df.sample(n=num_selections, random_state=0) | |
| 253 | final_sample = final_sample.append(sample) | |
| 254 | return sample | |
| 255 | ||
| 256 | # Sample for 3 selections per month | |
| 257 | sample_size = 3 | |
| 258 | sample = monthly_stratified_sample(df, 'Date of Sale', sample_size) | |
| 259 | sample.to_excel(file_output) | |
| 260 | ``` | |
| 261 | ||
| 262 | ## Documenting the Results | |
| 263 | ||
| 264 | Once you've generated a proper sample, there are a few things left to do in | |
| 265 | order to properly ensure your process is reproducible. | |
| 266 | ||
| 267 | 1. Document the sample. Make sure the resulting file is readable and includes | |
| 268 | the documentation listed in the next bullet. | |
| 269 | 2. Include documentation around the data source, extraction techniques, any | |
| 270 | modifications made to the data, and be sure to include a copy of the script | |
| 271 | itself. | |
| 272 | 3. Whenever possible, perform a completeness and accuracy test to ensure your | |
| 273 | sample is coming from a complete and accurate population. To ensure | |
| 274 | completeness, compare the record count from the data source to the record | |
| 275 | count loaded into Python. To ensure accuracy, test a small sample against the | |
| 276 | source data (e.g., test 5 sales against the database to see if the details are | |
| 277 | accurate). | |
blog/2021-10-09-apache-redirect.org added +45
| @@ -0,0 +1,45 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-10-10 | |
| 3 | title = "Apache Redirect HTML Files to a Directory" | |
| 4 | description = "Learn how to conveniently rewrite certain Apache web file URLs to directories with the Apache web server." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## The Problem | |
| 9 | ||
| 10 | After recently switching static site generators (SSG), my blog URLs changed with | |
| 11 | no option to preserve the classic `.html` extension at the end of my blog | |
| 12 | post URLs. | |
| 13 | ||
| 14 | I really disliked using my old SSG ([Jekyll](https://jekyllrb.com)) and prefer | |
| 15 | my new tool ([Zola](https://www.getzola.org)) much more, so I was determined to | |
| 16 | figure out a way to get the proper redirect set up so that people who find my | |
| 17 | posts online aren't constantly met by 404 errors. | |
| 18 | ||
| 19 | ## The Solution | |
| 20 | ||
| 21 | To solve this problem, I really needed to solve two pieces: | |
| 22 | ||
| 23 | 1. Redirect all blog post URL requests from `/blog/some-post.html` to | |
| 24 | `/blog/some-post/`. | |
| 25 | 2. Ensure that no other `.html` files are redirected, such as `index.html`. | |
| 26 | ||
| 27 | After _a lot_ of tweaking and testing, I believe I have finally found the | |
| 28 | solution. The solution is shown below. | |
| 29 | ||
| 30 | ```config | |
| 31 | RewriteEngine On | |
| 32 | RewriteCond %{REQUEST_URI} !\index.html$ [NC] | |
| 33 | RewriteRule ^(.*).html$ https://example.com/$1 [R=301,L] | |
| 34 | ``` | |
| 35 | ||
| 36 | This piece of code in the Apache `.conf` or `.htaccess` file will do the | |
| 37 | following: | |
| 38 | ||
| 39 | 1. Turn on the RewriteEngine so that we can modify URLs. | |
| 40 | 2. Ignore any `index.html` files from the rule we are about to specify. | |
| 41 | 3. Find any `.html` files within the website directory and redirect it to | |
| 42 | exclude the file extension. | |
| 43 | 4. The final piece is adding the trailing slash (`/`) at the end of the URL - | |
| 44 | you'll notice that I don't have an Apache rule for that since Apache handles | |
| 45 | that automatically. | |
blog/2021-12-04-cisa.org added +197
| @@ -0,0 +1,197 @@ | ||
| 1 | +++ | |
| 2 | date = 2021-12-04 | |
| 3 | title = "I Passed the CISA!" | |
| 4 | description = "After nearly a year of on-and-off studying, I passed the CISA exam by ISACA." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## What is the CISA? | |
| 9 | ||
| 10 | For those of you lucky enough not to be knee-deep in the world of IT/IS | |
| 11 | Auditing, [CISA](https://www.isaca.org/credentialing/cisa) stands for Certified | |
| 12 | Information Systems Auditor. This certification and exam are part of ISACA's | |
| 13 | suite of certifications. As I often explain it to people like my family, it | |
| 14 | basically means you're employed to use your knowledge of information systems, | |
| 15 | regulations, common threats, risks, etc. in order to assess an organization's | |
| 16 | current control of their risk. If a risk isn't controlled (and the company | |
| 17 | doesn't want to accept the risk), an IS auditor will suggest implementing a | |
| 18 | control to address that risk. | |
| 19 | ||
| 20 | Now, the CISA certification itself is, in my opinion, the main certification for | |
| 21 | this career. While certifications such as the CPA or CISSP are beneficial, | |
| 22 | nothing matches the power of the CISA for an IS auditor when it comes to getting | |
| 23 | hired, getting a raise/bonus, or earning respect in the field. | |
| 24 | ||
| 25 | However, to be honest, I am a skeptic of most certifications. I understand the | |
| 26 | value they hold in terms of how much you need to commit to studying or learning | |
| 27 | on the job, as well as the market value for certifications such as the CISA. | |
| 28 | But I also have known some very ~~incompetent~~ _less than stellar_ auditors | |
| 29 | who have CPAs, CISAs, CIAs, etc. | |
| 30 | ||
| 31 | The same goes for most industries: if a person is good at studying, they can | |
| 32 | earn the certification. However, that knowledge means nothing unless you're | |
| 33 | actually able to use it in real life and perform as expected of a certification | |
| 34 | holder. The challenge comes when people are hired or connected strictly because | |
| 35 | of their certifications or resume; you need to see a person work before you can | |
| 36 | assume them having a CISA means they're better than someone without the CISA. | |
| 37 | ||
| 38 | Okay, rant over. Certifications are generally accepted as a measuring stick of | |
| 39 | commitment and quality of an employee, so I am accepting it too. | |
| 40 | ||
| 41 | ## Exam Content | |
| 42 | ||
| 43 | The CISA is broken down into five sections, each weighted with a percentage of | |
| 44 | test questions that may appear. | |
| 45 | ||
| 46 |  | |
| 47 | ||
| 48 | Since the exam contains 150 questions, here's how those sections break down: | |
| 49 | ||
| 50 | | Exam Section | Percentage of Exam | Questions | | |
| 51 | |:---------------:|:------------------:|:---------:| | |
| 52 | | 1 | 21% | 32 | | |
| 53 | | 2 | 17% | 26 | | |
| 54 | | 3 | 12% | 18 | | |
| 55 | | 4 | 23% | 34 | | |
| 56 | | 5 | 27% | 40 | | |
| 57 | | **Grand Total** | **100%** | **150** | | |
| 58 | ||
| 59 | ## My Studying Habits | |
| 60 | ||
| 61 | This part is a little hard for me to break down into specific detail due to the | |
| 62 | craziness of the last year. While I officially purchased my studying materials | |
| 63 | in December 2020 and opened them to "start studying" in January 2021, I really | |
| 64 | wasn't able to study much due to the demands of my job and personal life. | |
| 65 | ||
| 66 | Let me approach this from a few different viewpoints. | |
| 67 | ||
| 68 | ### Study Materials | |
| 69 | ||
| 70 | Let's start by discussing the study materials I purchased. I'll be referring to | |
| 71 | #1 as the CRM and #2 as the QAE. | |
| 72 | ||
| 73 | 1. [CISA Review Manual, 27th Edition | Print](https://store.isaca.org/s/store#/store/browse/detail/a2S4w000004KoCbEAK) | |
| 74 | 2. [CISA Review Questions, Answers & Explanations Manual, 12th Edition | Print](https://store.isaca.org/s/store#/store/browse/detail/a2S4w000004KoCcEAK) | |
| 75 | ||
| 76 | The CRM is an excellent source of information and could honestly be used as a | |
| 77 | reference for most IS auditors as a learning reference during their daily audit | |
| 78 | responsibilities. However, it is **full** of information and can be overloading | |
| 79 | if you're not good at filtering out useless information while studying. | |
| 80 | ||
| 81 | The QAE is the real star of the show here. This book contains 1000 questions, | |
| 82 | separated by exam section, and a practice exam. My only complaint about the QAE | |
| 83 | is that each question is immediately followed with the correct answer and | |
| 84 | explanations below it, which means I had to use something to constantly cover | |
| 85 | the answers while I was studying. | |
| 86 | ||
| 87 | I didn't use the online database version of the QAE, but I've heard that it's | |
| 88 | easier to use than the printed book. However, it is more expensive ($299 | |
| 89 | database vs $129 book) which might be important if you're paying for materials | |
| 90 | yourself. | |
| 91 | ||
| 92 | In terms of question difficulty, I felt that the QAE was a good representation | |
| 93 | of the actual exam. I've seen a lot of people online say it wasn't accurate to | |
| 94 | the exam or that it was much easier/harder, but I disagree with all of those. | |
| 95 | The exam was fairly similar to the QAE, just focusing on whichever topics they | |
| 96 | chose for my version of the exam. | |
| 97 | ||
| 98 | If you understand the concepts, skim the CRM (and read in-depth on topics you | |
| 99 | struggle with), and use the QAE to continue practicing exam-like questions, you | |
| 100 | should be fine. I didn't use any online courses, videos, etc. - the ISACA | |
| 101 | materials are more than enough. | |
| 102 | ||
| 103 | ### Studying Process | |
| 104 | ||
| 105 | While I was able to briefly read through sections 1 and 2 in early 2021, I had | |
| 106 | to stop and take a break from February/March to September. | |
| 107 | I switched jobs in September, which allowed me a lot more free time to study. | |
| 108 | ||
| 109 | In September, I studied sections 3-5, took notes, and did a quick review of the | |
| 110 | section topics. Once I felt comfortable with my notes, I took a practice exam | |
| 111 | from the QAE manual and scored 70% (105/150). | |
| 112 | ||
| 113 | Here's a breakdown of my initial practice exam: | |
| 114 | ||
| 115 | | Exam Section | Incorrect | Correct | Grand Total | Percent | | |
| 116 | |:---------------:|:---------:|:-------:|:-----------:|:-------:| | |
| 117 | | 1 | 8 | 25 | 33 | 76% | | |
| 118 | | 2 | 5 | 20 | 25 | 80% | | |
| 119 | | 3 | 6 | 12 | 18 | 67% | | |
| 120 | | 4 | 10 | 23 | 33 | 70% | | |
| 121 | | 5 | 16 | 25 | 41 | 61% | | |
| 122 | | **Grand Total** | **45** | **105** | **150** | **70%** | | |
| 123 | ||
| 124 | As I expected, my toughest sections were related to project management, | |
| 125 | development, implementation, and security. | |
| 126 | ||
| 127 | This just leaves October and November. For these months, I tried to practice | |
| 128 | every few days, doing 10 questions for each section, until the exam. This came | |
| 129 | out to 13 practice sessions, ~140 questions per section, and ~700 questions | |
| 130 | total. | |
| 131 | ||
| 132 | While some practice sessions were worse and some were better, the final results | |
| 133 | were similar to my practice exam results. As you can see below, my averages were | |
| 134 | slightly worse than my practice exam. However, I got in over 700 questions of | |
| 135 | practice and, most importantly, **I read through the explanations every time I | |
| 136 | answered incorrectly and learned from my mistakes**. | |
| 137 | ||
| 138 | | Exam Section | Incorrect | Correct | Grand Total | Percent | | |
| 139 | |:---------------:|:---------:|:-------:|:-----------:|:-------:| | |
| 140 | | 1 | 33 | 108 | 141 | 77% | | |
| 141 | | 2 | 33 | 109 | 142 | 77% | | |
| 142 | | 3 | 55 | 89 | 144 | 62% | | |
| 143 | | 4 | 52 | 88 | 140 | 63% | | |
| 144 | | 5 | 55 | 85 | 140 | 61% | | |
| 145 | | **Grand Total** | **228** | **479** | **707** | **68%** | | |
| 146 | ||
| 147 |  | |
| 148 | ||
| 149 | ## Results | |
| 150 | ||
| 151 | Now, how do the practice scores reflect my actual results? | |
| 152 | After all, it's hard to tell how good a practice regimen is unless you see | |
| 153 | how it turns out. | |
| 154 | ||
| 155 | | Exam Section | Section Name | Score | | |
| 156 | |:------------:|------------------------------------------------------------------|:-------:| | |
| 157 | | 1 | Information Systems Auditing Process | 678 | | |
| 158 | | 2 | Governance and Management of IT | 590 | | |
| 159 | | 3 | Information Systems Acquisition, Development, and Implementation | 721 | | |
| 160 | | 4 | Information Systems Operations and Business Resilience | 643 | | |
| 161 | | 5 | Protection of Information Assets | 511 | | |
| 162 | | **TOTAL** | | **616** | | |
| 163 | ||
| 164 | Now, in order to pass the CISA, you need at least 450 on a sliding scale of | |
| 165 | 200-800. Personally, I really have no clue what an average CISA score is. After | |
| 166 | a _very_ brief look online, I can see that the high end is usually in the low | |
| 167 | 700s. In addition, only about 50-60% of people pass the exam. | |
| 168 | ||
| 169 | Given this information, I feel great about my scores. | |
| 170 | 616 may not be phenomenal, and I wish I had done better on sections 2 & 5, | |
| 171 | but my practicing seems to have worked very well overall. | |
| 172 | ||
| 173 | However, the practice results do not conform to the actual results. Section 2 | |
| 174 | was one of my highest practice sections and was my second-lowest score in the | |
| 175 | exam. Conversely, section 3 was my second-lowest practice section and turned out | |
| 176 | to be my highest actual score! | |
| 177 | ||
| 178 | After reflecting, it is obvious that if you have any background on the CISA | |
| 179 | topics at all, the most important part of studying is doing practice questions. | |
| 180 | You really need to understand how to read the questions critically and pick the | |
| 181 | best answer. | |
| 182 | ||
| 183 | ## Looking Forward | |
| 184 | ||
| 185 | I am extremely happy that I was finally able to pass the CISA. Looking to the | |
| 186 | future, I'm not sure what's next in terms of professional learning. My current | |
| 187 | company offers internal learning courses, so I will most likely focus on that if | |
| 188 | I need to gain more knowledge in certain areas. | |
| 189 | ||
| 190 | To be fair, even if you pass the CISA, it's hard to become an expert on any | |
| 191 | specific topic found within. | |
| 192 | My career may take me in a different direction, and I might need to focus | |
| 193 | more on security or networking certifications (or possibly building a better | |
| 194 | analysis/visualization portfolio if I want to go into data analysis/science). | |
| 195 | ||
| 196 | All I know is that I am content at the moment and extremely proud of my | |
| 197 | accomplishment. | |
blog/2022-02-10-leaving-the-office.org added +227
| @@ -0,0 +1,227 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-02-10 | |
| 3 | title = "Leaving Office-Based Work In the Past" | |
| 4 | description = "As the world moves on from the before-times, I'm taking a look back on how working from different office environments affected my effectiveness and comfort." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## The Working World is Changing | |
| 9 | ||
| 10 | There has been a trend for the past few years of companies slowly realizing that | |
| 11 | the pandemic is not just a temporary state that will go away eventually and let | |
| 12 | everything return to the way it was before. In terms of business and employment, | |
| 13 | this means that more and more jobs are being offered as permanently remote | |
| 14 | roles. | |
| 15 | ||
| 16 | I had always dreamt of working from home but thought of it as a fantasy, | |
| 17 | especially since I did not want to move over into the software development | |
| 18 | field. However, I have found that almost all roles being sent to me via | |
| 19 | recruiters are permanently remote (although most are limited to US citizens or | |
| 20 | even region-locked for companies who only operate in select states). | |
| 21 | ||
| 22 | I decided to take a look back at my relatively short career so far and compare | |
| 23 | the positive and negative effects of the different work environments I've been | |
| 24 | in. | |
| 25 | ||
| 26 | ## In-Person Offices | |
| 27 | ||
| 28 | ### Retail Internship | |
| 29 | ||
| 30 | I started my first job as a management intern at a busy retail pharmacy, working | |
| 31 | my 40-hour weeks on my feet. | |
| 32 | As these retail stores don't believe in resting or sitting down, you can | |
| 33 | guarantee that you will spend entire shifts standing, walking, or running | |
| 34 | around the store. | |
| 35 | Unfortunately, I worked at a time when our store didn't have enough managers, | |
| 36 | so I spent the majority of my tenure at the store running and breaking a sweat. | |
| 37 | ||
| 38 | Now, things aren't all bad in retail stores like this. It is definitely tiring | |
| 39 | and inefficient to force employees to work constantly, or pretend to work if | |
| 40 | there's nothing to do, and not allow anyone to sit down. However, if you are | |
| 41 | able to operate a retail store with a limited crew and provide enough comfort | |
| 42 | and support, I believe these jobs could be both comfortable and efficient. | |
| 43 | ||
| 44 | ### Semi-Private Cubicles | |
| 45 | ||
| 46 |  | |
| 47 | ||
| 48 | After about a year, I was able to find another internship - this time, it was in | |
| 49 | my field of interest: internal auditing. This was for a life insurance company | |
| 50 | that was well over 100 years old. The age of the company shows if you work | |
| 51 | there, as most people in management are well into their 40s-60s with little to | |
| 52 | no youthful leadership in the company. | |
| 53 | Likewise, they owned a large headquarters in a nice area of town with plenty | |
| 54 | of space, parking, etc. | |
| 55 | ||
| 56 | One upside is that each person gets their own large L-shaped desk, formed into | |
| 57 | cubicles that house 4 desks/employees. These "pods" of 4-person cubicles are | |
| 58 | linked throughout each floor of the headquarters (except the sales | |
| 59 | people, who had that open-floor concept going on). The walls of the cubicle were | |
| 60 | tall and provided a lot of privacy and sound-proofing, except when I used the | |
| 61 | standing desk feature (I'm over 6 feet tall, so probably not an issue for most | |
| 62 | people). | |
| 63 | ||
| 64 | I loved this environment, it allowed me to focus on my work with minimal | |
| 65 | distractions, but also allowed easy access, so I could spin around in my chair | |
| 66 | and chat with my friends without leaving my chair. This is the closest I've been | |
| 67 | to a home office environment (which is my personal favorite, as I'll get to | |
| 68 | later in this post). | |
| 69 | ||
| 70 | ### Semi-Open Floor Concept | |
| 71 | ||
| 72 |  | |
| 73 | ||
| 74 | When I shifted to my first full-time internal audit job out of college, I was | |
| 75 | working at a company that was headquartered on a floor in a downtown high-rise | |
| 76 | building. The company was only about 20 years old when I worked there and were | |
| 77 | trying a lot of new things to attract young talent, one of which was a | |
| 78 | semi-open floor concept for the office. My department worked just around the | |
| 79 | hallway corner from the executive offices and used that "modern" layout young | |
| 80 | tech companies started using in the 2000s/2010s. | |
| 81 | ||
| 82 | Each desk was brief, and you could look most coworkers in the face without | |
| 83 | moving from your chair, I hated this so much. | |
| 84 | Directly to my left was the Chief Audit Executive (our department's leading | |
| 85 | boss), and his desk was pointed so that his face would stare straight at my | |
| 86 | desk all day. | |
| 87 | I spent more time thinking about who was looking at me or checking on me | |
| 88 | than actually working. | |
| 89 | ||
| 90 | The other annoying part of the open concept they used was that the kitchen area | |
| 91 | and pathways were too close to everyone's desks (since the desks were spread | |
| 92 | out, to provide space or something), so noise and conversation would be constant | |
| 93 | throughout the day while you try to work. For someone like me, who needs silence | |
| 94 | to get work done, that was a non-starter. | |
| 95 | ||
| 96 | ### Hotel Office Concept | |
| 97 | ||
| 98 |  | |
| 99 | ||
| 100 | I currently work for a company remotely (for now) and travel to the office every | |
| 101 | once in a while for events and to help coach the staff underneath me. The office | |
| 102 | I visit uses the hotel desk concept, where you need to check in at a touch | |
| 103 | screen when you enter the office and "rent" a desk for the day. The same goes | |
| 104 | for offices and meeting rooms. | |
| 105 | ||
| 106 | These desks are flat-top only and do not have any walls at all. In addition, | |
| 107 | they're stacked with one row of 4 desks facing another row of 4 desks. These | |
| 108 | pairs of desk rows are repeated through the office. | |
| 109 | ||
| 110 | This means that when I go, I need to rent a random desk or try to remember the | |
| 111 | unique ID numbers on desks I like. Once I rent it, I have to make sure no one | |
| 112 | sat down in that desk without renting it. Then, I can sit down and work, but | |
| 113 | will probably need to adjust the monitors so that I'm not staring in the face of | |
| 114 | the person across from me all day. Finally, I need to wear headphones as this | |
| 115 | environment does nothing to provide you with peace or quiet. | |
| 116 | ||
| 117 | Luckily, you can rent offices with doors that offer quiet and privacy, which can | |
| 118 | be very nice if you have a lot of meetings or webinars on a certain day. | |
| 119 | ||
| 120 | ## Home Office | |
| 121 | ||
| 122 |  | |
| 123 | ||
| 124 | Okay, now let's finally get to the home office concept. I have worked from home | |
| 125 | for a little over two years at this point, across three different | |
| 126 | jobs/employers. Over this time, I have experimented with a plethora of different | |
| 127 | organizational ideas, desks, and room layouts to find what works best for me. | |
| 128 | ||
| 129 | These things might not apply to you, and that's fine. | |
| 130 | Everyone has a different situation, and I really don't think you'll know | |
| 131 | what works until you try. | |
| 132 | ||
| 133 | ### Tip #1 | |
| 134 | ||
| 135 | Let's start with my top rule for a home office: | |
| 136 | ||
| 137 | > If you live with others, working in a shared space is not effective. | |
| 138 | ||
| 139 | It just does not work. | |
| 140 | If you have another person sleeping in your bedroom, it is difficult to | |
| 141 | manage your work schedule with their sleeping/work/school schedule. | |
| 142 | If they wake up after you need to start work, you might wake them up | |
| 143 | or have to suffer the agony of staring at bright screens in a dark room. | |
| 144 | ||
| 145 | In a similar vein, working from a location such as the living room likely won't | |
| 146 | work either. | |
| 147 | Distractions will come far more frequently: televisions, cooking, cleaning, | |
| 148 | deliveries, etc. If you're like me, you'll end up playing a game instead of | |
| 149 | actually doing any work. | |
| 150 | ||
| 151 | ### Tip #2 | |
| 152 | ||
| 153 | Okay, the second thing I've discovered that works for me: | |
| 154 | ||
| 155 | > Use the pomodoro method (or something similar) to balance work tasks with | |
| 156 | > personal tasks. | |
| 157 | ||
| 158 | I use a very casual version of the pomodoro method where I will work for 1-2 | |
| 159 | hours (usually set in strict intervals like 1, 1.5, 2 hours) and then will allow | |
| 160 | myself 30-60 minutes for personal tasks. This schedule works for me, since my | |
| 161 | work schedule really only comes to 3-6 hours of work per day. | |
| 162 | ||
| 163 | In this case, I'll work through my list of tasks for an hour or two and then | |
| 164 | give myself personal time to get drinks and food, wash dishes, put clothes in | |
| 165 | the washer, get the mail, etc. If you're in a convenient location, this usually | |
| 166 | gives time for things like getting groceries (as long as you're not a slow | |
| 167 | shopper). | |
| 168 | ||
| 169 | ### Tip #3 | |
| 170 | ||
| 171 | While I listed this one as number three, I don't think I'd accomplish anything | |
| 172 | without it: | |
| 173 | ||
| 174 | > Document everything: even things you didn't before - such as task lists and | |
| 175 | > notes from casual calls or meetings. | |
| 176 | ||
| 177 | I've noticed that staying in an office gave me more constant reminders of | |
| 178 | outstanding tasks or facts I had learned in a conversation. Translating | |
| 179 | everything to a digital world has made me lose a bit of that focus (perhaps | |
| 180 | since I don't have visual reminders?). | |
| 181 | ||
| 182 | Keeping a running task list of all things I have to do - even potential tasks! - | |
| 183 | has helped me keep up without missing anything small. Likewise, keeping notes | |
| 184 | for ALL meetings and calls, no matter how casual/quick, has improved my | |
| 185 | retention immensely. Beyond helping my mental recollection, it has saved me | |
| 186 | numerous times when I need to do a keyword search for some topic that was discussed | |
| 187 | 6+ months ago. | |
| 188 | ||
| 189 | ### Tip #4 | |
| 190 | ||
| 191 | Okay, last one for now. | |
| 192 | ||
| 193 | > Keep your work area clean. | |
| 194 | ||
| 195 | This one is straightforward, but I know some people struggle with cleanliness or | |
| 196 | may not believe it makes a difference. Trust me, keeping your desk area clean | |
| 197 | and organized makes a huge difference, both mentally and emotionally. | |
| 198 | ||
| 199 | Just think about it, you walk into your home office and see a clean desk with a | |
| 200 | laptop, dock, monitors, keyboard, mouse, and a notepad with a pen on top. | |
| 201 | ||
| 202 | Now imagine the opposite, there's an office with the same equipment, but there | |
| 203 | are clothes hanging on the chair, empty drink bottles, candy wrappers and dirty | |
| 204 | plates. This can take both a mental and emotional toll by bringing constant | |
| 205 | disarray and stress into your working environment. | |
| 206 | ||
| 207 | Just keep things clean each day, and you won't need to do any big cleaning days | |
| 208 | to recover. | |
| 209 | ||
| 210 | ## My Preferences | |
| 211 | ||
| 212 | I've talked about the different environments I've worked in and expressed some | |
| 213 | honest thoughts on pros or cons to each, but what do I prefer? Well, if you're | |
| 214 | reading along, you should be able to tell that I much prefer a home office above | |
| 215 | all else. | |
| 216 | ||
| 217 | Being able to control my own day and allot my time as needed has brought a | |
| 218 | calmness to my life and has allowed me to maximize each day. I feel far more | |
| 219 | effective and efficient in a home office than any other office, especially | |
| 220 | open-office layouts. | |
| 221 | ||
| 222 | If I do need to return to an office part-time in the future, I really hope the | |
| 223 | office will have privacy and quietness in order for me to get my work done. | |
| 224 | ||
| 225 | Cubicles are good! I agree with Alice (from the comic Dilbert): | |
| 226 | ||
| 227 |  | |
blog/2022-02-16-debian-and-nginx.org added +175
| @@ -0,0 +1,175 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-02-16 | |
| 3 | title = "Migrating to a New Web Server Setup with Debian, Nginx, and Agate" | |
| 4 | description = "I've finally taken the leap of faith on two different technologies for the first time: trying out Debian as my main server OS and Nginx as my web server software." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Server OS: Debian | |
| 9 | ||
| 10 |  | |
| 11 | ||
| 12 | I've used various Linux distributions throughout the years, but I've never used | |
| 13 | anything except Ubuntu for my servers. | |
| 14 | Why? | |
| 15 | I really have no idea, mostly just comfort around the commands and software | |
| 16 | availability. | |
| 17 | ||
| 18 | However, I have always wanted to try Debian as a server OS after testing it out | |
| 19 | in a VM a few years ago (side-note: I'd love to try Alpine too, but I always | |
| 20 | struggle with compatibility). So, I decided to launch a new VPS and use | |
| 21 | [Debian](https://www.debian.org) 11 as the OS. Spoiler alert: it feels identical | |
| 22 | to Ubuntu for my purposes. | |
| 23 | ||
| 24 | I did the normal things when first launching the VPS, such as adding a new user, | |
| 25 | locking down SSH, etc. If you want to see that level of detail, read my other | |
| 26 | post about | |
| 27 | [How to Set Up a VPS Web Server](https://0x4b1d.org/blog/how-to-set-up-a-vps-web-server/). | |
| 28 | ||
| 29 | All of this has been similar, apart from small things such as the location of | |
| 30 | users' home folders. No complaints at all from me - Debian seems great. | |
| 31 | ||
| 32 | ## Web Server: Nginx | |
| 33 | ||
| 34 |  | |
| 35 | ||
| 36 | Once I had the baseline server configuration set-up for Debian, I moved on to | |
| 37 | trying out [Nginx](https://nginx.org) as my web server software. This required | |
| 38 | me to install the `nginx` and `ufw` packages, as well as setting up the initial | |
| 39 | UFW config: | |
| 40 | ||
| 41 | ```sh | |
| 42 | sudo apt install nginx ufw | |
| 43 | sudo ufw allow 'Nginx Full' | |
| 44 | sudo ufw allow SSH | |
| 45 | sudo ufw enable | |
| 46 | sudo ufw status | |
| 47 | sudo systemctl status nginx | |
| 48 | ``` | |
| 49 | ||
| 50 | Once I had the firewall set, I moved on to creating the directories and files | |
| 51 | for my website. This is very easy and is basically the same as setting up an | |
| 52 | Apache server, so no struggles here. | |
| 53 | ||
| 54 | ```sh | |
| 55 | sudo mkdir -p /var/www/your_domain/html | |
| 56 | sudo chown -R $USER:$USER /var/www/your_domain/html | |
| 57 | sudo chmod -R 755 /var/www/your_domain | |
| 58 | nano /var/www/your_domain/html/index.html | |
| 59 | ``` | |
| 60 | ||
| 61 | The next part, creating the Nginx configuration files, is quite a bit different | |
| 62 | from Apache. First, you need to create the files in the `sites-available` folder | |
| 63 | and symlink it the `sites-enabled` folder. | |
| 64 | ||
| 65 | Creating the config file for your domain: | |
| 66 | ||
| 67 | ```sh | |
| 68 | sudo nano /etc/nginx/sites-available/your_domain | |
| 69 | ``` | |
| 70 | ||
| 71 | Default content for an Nginx config file: | |
| 72 | ||
| 73 | ```sh | |
| 74 | server { | |
| 75 | listen 80; | |
| 76 | listen [::]:80; | |
| 77 | ||
| 78 | root /var/www/your_domain/html; | |
| 79 | index index.html index.htm index.nginx-debian.html; | |
| 80 | ||
| 81 | server_name your_domain www.your_domain; | |
| 82 | ||
| 83 | location / { | |
| 84 | try_files $uri $uri/ =404; | |
| 85 | } | |
| 86 | } | |
| 87 | ``` | |
| 88 | ||
| 89 | Finally, symlink it together: | |
| 90 | ||
| 91 | ```sh | |
| 92 | sudo ln -s /etc/nginx/sites-available/your_domain /etc/nginx/sites-enabled/ | |
| 93 | ``` | |
| 94 | ||
| 95 | This will make your site available to the public (as long as you have | |
| 96 | `your_domain` DNS records pointed at the server's IP address)! | |
| 97 | ||
| 98 | Next, I used [certbot](https://certbot.eff.org/) to issue an HTTPS certificate | |
| 99 | for my domains using the following commands: | |
| 100 | ||
| 101 | ```sh | |
| 102 | sudo apt install snapd; sudo snap install core; sudo snap refresh core | |
| 103 | sudo snap install --classic certbot | |
| 104 | sudo ln -s /snap/bin/certbot /usr/bin/certbot | |
| 105 | sudo certbot --nginx | |
| 106 | ``` | |
| 107 | ||
| 108 | Now that certbot ran successfully and updated my Nginx config files to include a | |
| 109 | `443` server block of code, I went back in and edited the config file to include | |
| 110 | security HTTP headers. This part is optional, but is recommended for security | |
| 111 | purposes; you can even test a website's HTTP header security at | |
| 112 | [Security Headers](https://securityheaders.com/). | |
| 113 | ||
| 114 | The configuration below shows a set-up where you only want your website to serve | |
| 115 | content from its own domain, except for images and scripts, which may come from | |
| 116 | `nullitics.com`. All other content would be blocked from loading in a browser. | |
| 117 | ||
| 118 | ```sh | |
| 119 | sudo nano /etc/nginx/sites-available/your_domain | |
| 120 | ``` | |
| 121 | ||
| 122 | ```sh | |
| 123 | server { | |
| 124 | ... | |
| 125 | add_header Content-Security-Policy "default-src 'none'; img-src 'self' https://nullitics.com; script-src 'self' https://nullitics.com; style-src 'self'; font-src 'self'"; | |
| 126 | add_header X-Content-Type-Options "nosniff"; | |
| 127 | add_header X-XSS-Protection "1; mode=block"; | |
| 128 | add_header X-Frame-Options "DENY"; | |
| 129 | add_header Strict-Transport-Security "max-age=63072000; includeSubDomains"; | |
| 130 | add_header Referrer-Policy "no-referrer"; | |
| 131 | ... | |
| 132 | } | |
| 133 | ``` | |
| 134 | ||
| 135 | ```sh | |
| 136 | sudo systemctl restart nginx | |
| 137 | ``` | |
| 138 | ||
| 139 | ### Nginx vs. Apache | |
| 140 | ||
| 141 | As I stated at the beginning, my historical hesitation with trying Nginx was | |
| 142 | that the differences in configuration formats scared me away from leaving | |
| 143 | Apache. | |
| 144 | However, I prefer Nginx to Apache for a few reasons: | |
| 145 | ||
| 146 | 1. Nginx uses only one config file (`your_domain`) vs. Apache's two-file | |
| 147 | approach for HTTP vs. HTTPS (`your_domain.conf` and | |
| 148 | `your_domain-le-ssl.conf`). | |
| 149 | 2. Symlinking new configurations files and reloading Nginx are way easier than | |
| 150 | Apache's process of having to enable headers with `a2enmod mod_headers`, | |
| 151 | enable PHP with `a2enmod php` (plus any other mods you need), and then | |
| 152 | enabling sites with `a2ensite`, and THEN reloading Apache. | |
| 153 | 3. The contents of the Nginx config files seem more organized and logical with | |
| 154 | the curly-bracket approach. | |
| 155 | This is a minor reason, but everything just felt cleaner while I was | |
| 156 | installing my sites and that had a big quality of life impact on the | |
| 157 | installation for me. | |
| 158 | ||
| 159 | They're both great software packages, but Nginx just seems more organized and | |
| 160 | easier to use these days. | |
| 161 | I will certainly be exploring the Nginx docs to see what other fun things I | |
| 162 | can do with all of this. | |
| 163 | ||
| 164 | ## Gemini Server: Agate | |
| 165 | ||
| 166 |  | |
| 167 | ||
| 168 | Finally, I set up the Agate software on this server again to host my Gemini | |
| 169 | server content, using Rust as I have before. | |
| 170 | You can read my other post for more information on installing Agate: | |
| 171 | [Hosting a Gemini Server](https://0x4b1d.org/blog/hosting-a-gemini-server/). | |
| 172 | ||
| 173 | All in all, Debian + Nginx is very slick and I prefer it over my old combination | |
| 174 | of Ubuntu + Apache (although it's really just Nginx > Apache for me, since | |
| 175 | Debian seems mostly the same as Ubuntu is so far). | |
blog/2022-02-17-exiftool.org added +63
| @@ -0,0 +1,63 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-02-17 | |
| 3 | title = "Stripping Image Metadata with exiftool" | |
| 4 | description = "Become a h@ck3r (i.e., just protect your privacy) by using the exiftool command line package to strip metadata from images." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Why Strip Metadata? | |
| 9 | ||
| 10 | Okay, so you want to strip metadata from your photos. Perhaps you take pictures | |
| 11 | of very rare birds, and the location metadata is a gold mine for poachers, or | |
| 12 | perhaps you're just privacy-oriented like me and prefer to strip metadata from | |
| 13 | publicly-available images. | |
| 14 | ||
| 15 | There are various components of image metadata that you may want to delete | |
| 16 | before releasing a photo to the public. Here's an incomplete list of things I | |
| 17 | could easily see just by inspecting a photo on my laptop: | |
| 18 | ||
| 19 | - Location (Latitude & Longitude) | |
| 20 | - Dimensions | |
| 21 | - Device Make & Model | |
| 22 | - Color Space | |
| 23 | - Color Profile | |
| 24 | - Focal Length | |
| 25 | - Alpha Channel | |
| 26 | - Red Eye | |
| 27 | - Metering Mode | |
| 28 | - F Number | |
| 29 | ||
| 30 | Regardless of your reasoning, I'm going to explain how I used the `exiftool` | |
| 31 | package in Linux to automatically strip metadata from all images in a directory | |
| 32 | (+ subdirectories). | |
| 33 | ||
| 34 | ## Installing `exiftool` | |
| 35 | ||
| 36 | First things first: we need to install the tool. I'm running Debian 11 on my | |
| 37 | server (Ubuntu will work the same), so the command is as simple as: | |
| 38 | ||
| 39 | ```sh | |
| 40 | sudo apt install exiftool | |
| 41 | ``` | |
| 42 | ||
| 43 | There are different tools that can accomplish the same thing across | |
| 44 | distributions, but I really only care to test out this one package. | |
| 45 | ||
| 46 | ## Recursively Strip Data | |
| 47 | ||
| 48 | I actually use this tool extensively to strip any photos uploaded to the website | |
| 49 | that serves all the images for my blog (`img.0x4b1d.org`). | |
| 50 | ||
| 51 | The following command is incredibly useful and can be modified to include any | |
| 52 | image extensions that `exiftool` supports: | |
| 53 | ||
| 54 | ```sh | |
| 55 | exiftool -r -all= -ext jpg -ext png /path/to/directory/ | |
| 56 | ``` | |
| 57 | ||
| 58 | See below for the results of my most recent usage of `exiftool` after I uploaded | |
| 59 | the image for this blog post. You can see that the command will let you know how | |
| 60 | many directories were scanned, how many images were updated, and how many images | |
| 61 | were unchanged. | |
| 62 | ||
| 63 |  | |
blog/2022-02-20-nginx-caching.org added +67
| @@ -0,0 +1,67 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-02-20 | |
| 3 | title = "Caching Static Content with Nginx" | |
| 4 | description = "Learn how to cache static content client-side with Nginx configurations." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Update Your Nginx Config to Cache Static Files | |
| 9 | ||
| 10 | If you run a website on Nginx that serves static content (i.e., content that is | |
| 11 | not dynamic and changing with interactions from the user), you would likely | |
| 12 | benefit from caching that content on the client-side. If you're used to Apache | |
| 13 | and looking for the Nginx equivalent, this post should help. | |
| 14 | ||
| 15 | Luckily, setting up the cache is as easy as identifying the file types you want | |
| 16 | to cache and determining the expiration length. To include more file types, | |
| 17 | simply use the bar separator (`|`) and type the new file extension you want to | |
| 18 | include. | |
| 19 | ||
| 20 | ```config | |
| 21 | server { | |
| 22 | ... | |
| 23 | ||
| 24 | location ~* .(css|js|jpg|jpeg|gif|png|ico)$ { | |
| 25 | expires 30d; | |
| 26 | } | |
| 27 | ||
| 28 | ... | |
| 29 | } | |
| 30 | ``` | |
| 31 | ||
| 32 | I have seen some people who prefer to set `expires` as `365d` or even `max`, but | |
| 33 | that is only for stable, infrequently changing websites. | |
| 34 | As my site often changes (i.e., I'm never content with my website), I need | |
| 35 | to know that my readers are seeing the new content without waiting too long. | |
| 36 | ||
| 37 | So, I went ahead and set the expiration date at `30d`, which is short enough to | |
| 38 | refresh for readers but long enough that clients/browsers won't be re-requesting | |
| 39 | the static files too often, hopefully resulting in faster loading times, as | |
| 40 | images should be the only thing slowing down my site. | |
| 41 | ||
| 42 | ## Testing Results | |
| 43 | ||
| 44 | To test my changes to the Nginx configuration, I used the | |
| 45 | [HTTP Header Live](https://addons.mozilla.org/en-US/firefox/addon/http-header-live/) | |
| 46 | extension on my Gecko browser and used the sidebar to inspect the headers of a | |
| 47 | recent image from my blog. | |
| 48 | ||
| 49 | In the image below, you can see that the `Cache-Control` header is now present | |
| 50 | and set to 2592000, which is 30 days represented in seconds (30 days _ 24 | |
| 51 | hours/day _ 60 minutes/hour \* 60 seconds/minute = 2,592,000 seconds). | |
| 52 | ||
| 53 | The `Expires` field is now showing 22 March 2022, which is 30 days from the day | |
| 54 | of this post, 20 February 2022. | |
| 55 | ||
| 56 |  | |
| 57 | ||
| 58 | ## Caveats | |
| 59 | ||
| 60 | Remember that this caching system is **client-side**, which means that content | |
| 61 | is only cached for as long as a client allows it. For example, my browser purges | |
| 62 | all caches, data, etc. upon exit, so this caching policy will only work as long | |
| 63 | as my browser remains open and running. | |
| 64 | ||
| 65 | If you need to test updates to your site, you'll need to clear the cache to see | |
| 66 | updates for any file extension you configured. This can often be done with the | |
| 67 | `Shift + F5` or `Ctrl + F5` key combinations in most browsers. | |
blog/2022-02-22-tuesday.org added +35
| @@ -0,0 +1,35 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-02-22 | |
| 3 | title = "Tuesday" | |
| 4 | description = "Today is Tuesday, February 22nd, 2022 - also a ubiquitous six-digit palindrome date known as Twosday." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Tuesday, Twosday | |
| 9 | ||
| 10 | I'm taking a break from my usual technology-related posts and writing about | |
| 11 | something a little more enjoyable today. | |
| 12 | ||
| 13 | Today is Tuesday, February 22nd, 2022. Today is 02-22-2022. Today is Twosday. | |
| 14 | ||
| 15 | Let's look at everything that fell in place today: | |
| 16 | ||
| 17 | 1. Written in the `m-dd-yy` or `dd-m-yy` formats, today is 2-22-22 or 22-2-22, | |
| 18 | which is a neat little palindrome in either format. (The last ubiquitous | |
| 19 | six-digit palindrome was 1-11-11.) | |
| 20 | 2. Today is Tuesday, which is why everyone is using the nickname Twosday to call | |
| 21 | out these similarities. | |
| 22 | 3. Falling on Tuesday means today is the 2nd day of the week (for most | |
| 23 | cultures. For the US, it's the 3rd day of the week since we start on | |
| 24 | Sunday). | |
| 25 | 4. The only culture I could find with a connection to a `2` is that some Slavic | |
| 26 | languages derived their version of Tuesday from the Old Church Slavonic word | |
| 27 | `въторъ`, meaning "the second." | |
| 28 | 5. Written in the classic monospaced, digital font (think of digital clocks from | |
| 29 | the 80s/90s), there is nice symmetry to the numbers | |
| 30 | ([view the image here](https://img.0x4b1d.org/blog/20220222-tuesday/digital_font.png)!). | |
| 31 | 6. This one isn't naturally-occurring, but it seems people around the world are | |
| 32 | celebrating the day. For example, a group is putting together | |
| 33 | [a wedding of 222 couples at the California State Capitol in Sacramento](https://www.eventbrite.com/e/2-22-22-a-collective-wedding-ceremony-at-the-state-capitol-tickets-211434605597), | |
| 34 | concluding at exactly 2:22 PM. These couples will record their marriage dates | |
| 35 | as 2-22-22 2:22 PM. Tickets were on sale for $222.22. | |
blog/2022-03-02-note-taking.org added +125
| @@ -0,0 +1,125 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-03-02 | |
| 3 | title = "Easy, Reliable Note-Taking" | |
| 4 | description = "After years of testing, I have finally settled on maintaining my collection of personal notes and writings." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Choosing Durable File Formats | |
| 9 | ||
| 10 | > TL;DR: Write in a format that can be easily rendered and read in plain-text | |
| 11 | > mode (e.g., `.txt`, `.md`, etc.). | |
| 12 | ||
| 13 | As I've written more and more over the years, I've found that my love of | |
| 14 | note-taking is always growing. Everything I learn or need to remember can be | |
| 15 | written down in a note and saved digitally, with no cost to myself. Unlike paper | |
| 16 | copies that need physical storage space, digital files simply need space on your | |
| 17 | local disk or cloud storage, which is fairly abundant these days. | |
| 18 | ||
| 19 | However, I had a historical struggle with migration of notes between different | |
| 20 | apps that require different formats and applied proprietary styling. This meant | |
| 21 | that I had to go through each note during migration and edit the file to look | |
| 22 | presentable again. | |
| 23 | ||
| 24 | For the last year or two, I have written everything exclusively in | |
| 25 | [Markdown](https://en.wikipedia.org/wiki/Markdown) format. Small notes, | |
| 26 | long-form writings, and even these blog posts are all written in Markdown. | |
| 27 | ||
| 28 | Why Markdown? While I do appreciate the simplicity of plain-text files without | |
| 29 | any formatting, I often need visual cues such as heading and code blocks to keep | |
| 30 | my thoughts straight. Markdown provides a minimal set of styling indicators for | |
| 31 | me to style my notes without adding any proprietary, embedded data into the | |
| 32 | files. If I want a top-level heading, I simply add a hash (`#`) before the line. | |
| 33 | An added bonus is that even if a system doesn't understand Markdown, it will | |
| 34 | render it as plain-text and I can read it just as easily. | |
| 35 | ||
| 36 | For example, here's how TextEdit on macOS will open and display a Markdown file | |
| 37 | in plain-text, since it does contain any features to preview Markdown as HTML: | |
| 38 | ||
| 39 |  | |
| 40 | ||
| 41 | ## Saving & Syncing Files | |
| 42 | ||
| 43 | In order to read and edit my notes across platforms, I use my personal cloud | |
| 44 | storage through Tresorit due to its native integration with macOS and iOS file | |
| 45 | managers. In addition, Tresorit works well on Debian-based Linux distros, which | |
| 46 | I used before macOS (and will likely switch back to in a few years). | |
| 47 | ||
| 48 | You can use whatever sync software you want - syncing plain-text or markdown | |
| 49 | files is incredibly easy and fast, since the files are generally tiny. | |
| 50 | ||
| 51 | Since the cloud storage syncs files automatically, there is no need for me to | |
| 52 | sync anything manually or kick-off a sync job to update my files. This means | |
| 53 | that I can edit on mobile, and it takes about 5-10 seconds to see the changes on | |
| 54 | desktop. | |
| 55 | ||
| 56 | ### Version Control with Git | |
| 57 | ||
| 58 | A different approach I've contemplated is storing my notes and attachments is | |
| 59 | using a hosted Git repository to track changes to the files. However, I don't | |
| 60 | want to rely on an external service that could potentially see into my data, | |
| 61 | even if the repository is private. | |
| 62 | ||
| 63 | I might just do `git init` locally and then commit my changes each time I write | |
| 64 | or update a note, but that seems to be a lot of work just for tracking changes - | |
| 65 | which I don't necessarily care to know. | |
| 66 | ||
| 67 | ### Backups! | |
| 68 | ||
| 69 | One small addition to the storage/sync conversation is the idea of backups. | |
| 70 | Personally, I manually create periodic backups of my entire cloud storage, | |
| 71 | compress it into an archive, and store it on my home server. | |
| 72 | ||
| 73 | To improve my workflow, I am going to be exploring options to automatically | |
| 74 | compress the mounted cloud directory and send it over to my server on a set | |
| 75 | schedule. | |
| 76 | ||
| 77 | ## Writing on Desktop | |
| 78 | ||
| 79 | > **Update (06.14.22)**: Since writing this post, I have reverted to simply | |
| 80 | > keeping my `notes` folder open and opening notes individually in TextEdit | |
| 81 | > for a more minimal and relaxing writing experience on the desktop. | |
| 82 | ||
| 83 | The bulk of my writing occurs in a desktop environment, with a full keyboard | |
| 84 | layout and wide screen. I don't illustrate with a smart pen, I rarely use | |
| 85 | embedded images, and I love being able to see all of my notes/directories in a | |
| 86 | sidebar. | |
| 87 | ||
| 88 | With this simple set of requirements, I chose [Obsidian](https://obsidian.md) as | |
| 89 | my desktop text editor. | |
| 90 | Obsidian has some in-depth tools like a graph view, command palette, | |
| 91 | mentions, etc., but I've found that using it as a simple Markdown editor is | |
| 92 | incredibly easy and straightforward. | |
| 93 | ||
| 94 | Here's an example of how my Markdown notes look when opened in plain-text mode: | |
| 95 | ||
| 96 |  | |
| 97 | ||
| 98 | Here's the "live preview" version, where the Markdown is rendered into its HTML | |
| 99 | format: | |
| 100 | ||
| 101 |  | |
| 102 | ||
| 103 | ### Programming on Desktop | |
| 104 | ||
| 105 | While I was writing this, I realized I should specify that I don't use the same | |
| 106 | editor for writing notes and for writing code. For programming purposes, I use | |
| 107 | [VSCodium](https://vscodium.com) as my development IDE. | |
| 108 | ||
| 109 | ## Writing on Mobile | |
| 110 | ||
| 111 | Personally, I write very little on mobile, except when needing to take | |
| 112 | important notes on-the-go. Any long-form writing, journals, etc. are done at | |
| 113 | home, where I always have my laptop available. | |
| 114 | ||
| 115 | I wanted a simple and foolproof editor for iOS, preferably open-source. | |
| 116 | After a long journey of testing the few (& terrible) open-source iOS | |
| 117 | note-taking apps, I finally found a phenomenal one: | |
| 118 | [Runestone](https://github.com/simonbs/runestone). This app is fantastic | |
| 119 | for note-taking, has plenty of optional features, and integrates natively | |
| 120 | with the iOS file manager. | |
| 121 | ||
| 122 | This app opens the iOS file manager and allows you to click any file you want, | |
| 123 | opens it up in an editor, and lets me save and close out of that note. | |
| 124 | ||
| 125 | Quite simple but effective. | |
blog/2022-03-03-financial-database.org added +253
| @@ -0,0 +1,253 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-03-03 | |
| 3 | title = "Maintaining a Personal Financial Database" | |
| 4 | description = "Detailing my process for creating, maintaining, and utilizing a personal financial database." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Personal Financial Tracking | |
| 9 | ||
| 10 | For the last 6-ish years, I've tracked my finances in a spreadsheet. This is | |
| 11 | common practice in the business world, but any good dev will cringe at the | |
| 12 | thought of storing long-term data in a spreadsheet. A spreadsheet is not for | |
| 13 | long-term storage or as a source of data to pull data/reports. | |
| 14 | ||
| 15 | As I wanted to expand the functionality of my financial data (e.g., adding more | |
| 16 | reports), I decided to migrate the data into a database. To run reports, I would | |
| 17 | query the database and use a language like Python or Javascript to process the | |
| 18 | data, perform calculations, and visualize the data. | |
| 19 | ||
| 20 | ## SQLite | |
| 21 | ||
| 22 | When choosing the type of database I wanted to use for this project, I was split | |
| 23 | between three options: | |
| 24 | ||
| 25 | 1. MySQL: The database I have the most experience with and have used for years. | |
| 26 | 2. PostgreSQL: A database I'm new to, but want to learn. | |
| 27 | 3. SQLite: A database that I've used for a couple projects and have moderate | |
| 28 | experience. | |
| 29 | ||
| 30 | I ended up choosing SQLite since it can be maintained within a single `.sqlite` | |
| 31 | file, which allows me more flexibility for storage and backup. I keep this file | |
| 32 | in my cloud storage and pull it up whenever needed. | |
| 33 | ||
| 34 | ### GUI Editing | |
| 35 | ||
| 36 | Since I didn't want to try and import 1000–1500 records into my new database | |
| 37 | via the command line, I opted to use | |
| 38 | [DB Browser for SQLite (DB4S)](https://sqlitebrowser.org/) as a GUI tool. This | |
| 39 | application is excellent, and I don't see myself going back to the CLI | |
| 40 | when working in this database. | |
| 41 | ||
| 42 | DB4S allows you to copy a range of cells from a spreadsheet and paste it | |
| 43 | straight into the SQL table. I used this process for all 36 accounts, 1290 | |
| 44 | account statements, and 126 pay statements. Overall, I'm guessing this took | |
| 45 | anywhere between 4–8 hours. In comparison, it probably took me 2-3 days to | |
| 46 | initially create the spreadsheet. | |
| 47 | ||
| 48 |  | |
| 49 | ||
| 50 | ### Schema | |
| 51 | ||
| 52 | The schema for this database is actually extremely simple and involves only | |
| 53 | three tables (for now): | |
| 54 | ||
| 55 | 1. Accounts | |
| 56 | 2. Statements | |
| 57 | 3. Payroll | |
| 58 | ||
| 59 | **Accounts** | |
| 60 | ||
| 61 | The Accounts table contains summary information about an account, such as a car | |
| 62 | loan or a credit card. By viewing this table, you can find high-level data, such | |
| 63 | as interest rate, credit line, or owner. | |
| 64 | ||
| 65 | ```sql | |
| 66 | CREATE TABLE "Accounts" ( | |
| 67 | "AccountID" INTEGER NOT NULL UNIQUE, | |
| 68 | "AccountType" TEXT, | |
| 69 | "AccountName" TEXT, | |
| 70 | "InterestRate" NUMERIC, | |
| 71 | "CreditLine" NUMERIC, | |
| 72 | "State" TEXT, | |
| 73 | "Owner" TEXT, | |
| 74 | "Co-Owner" TEXT, | |
| 75 | PRIMARY KEY("AccountID" AUTOINCREMENT) | |
| 76 | ) | |
| 77 | ``` | |
| 78 | ||
| 79 | **Statements** | |
| 80 | ||
| 81 | The Statements table uses the same unique identifier as the Accounts table, | |
| 82 | meaning you can join the tables to find a monthly statement for any of the | |
| 83 | accounts listed in the Accounts table. Each statement has an account ID, | |
| 84 | statement date, and total balance. | |
| 85 | ||
| 86 | ```sql | |
| 87 | CREATE TABLE "Statements" ( | |
| 88 | "StatementID" INTEGER NOT NULL UNIQUE, | |
| 89 | "AccountID" INTEGER, | |
| 90 | "StatementDate" INTEGER, | |
| 91 | "Balance" NUMERIC, | |
| 92 | PRIMARY KEY("StatementID" AUTOINCREMENT), | |
| 93 | FOREIGN KEY("AccountID") REFERENCES "Accounts"("AccountID") | |
| 94 | ) | |
| 95 | ``` | |
| 96 | ||
| 97 | **Payroll** | |
| 98 | ||
| 99 | The Payroll table is a separate entity, unrelated to the Accounts or Statements | |
| 100 | tables. This table contains all information you would find on a pay statement | |
| 101 | from an employer. As you change employers or obtain new perks/benefits, just add | |
| 102 | new columns to adapt to the new data. | |
| 103 | ||
| 104 | ```sql | |
| 105 | CREATE TABLE "Payroll" ( | |
| 106 | "PaycheckID" INTEGER NOT NULL UNIQUE, | |
| 107 | "PayDate" TEXT, | |
| 108 | "Payee" TEXT, | |
| 109 | "Employer" TEXT, | |
| 110 | "JobTitle" TEXT, | |
| 111 | "IncomeRegular" NUMERIC, | |
| 112 | "IncomePTO" NUMERIC, | |
| 113 | "IncomeHoliday" NUMERIC, | |
| 114 | "IncomeBonus" NUMERIC, | |
| 115 | "IncomePTOPayout" NUMERIC, | |
| 116 | "IncomeReimbursements" NUMERIC, | |
| 117 | "FringeHSA" NUMERIC, | |
| 118 | "FringeStudentLoan" NUMERIC, | |
| 119 | "Fringe401k" NUMERIC, | |
| 120 | "PreTaxMedical" NUMERIC, | |
| 121 | "PreTaxDental" NUMERIC, | |
| 122 | "PreTaxVision" NUMERIC, | |
| 123 | "PreTaxLifeInsurance" NUMERIC, | |
| 124 | "PreTax401k" NUMERIC, | |
| 125 | "PreTaxParking" NUMERIC, | |
| 126 | "PreTaxStudentLoan" NUMERIC, | |
| 127 | "PreTaxOther" NUMERIC, | |
| 128 | "TaxFederal" NUMERIC, | |
| 129 | "TaxSocial" NUMERIC, | |
| 130 | "TaxMedicare" NUMERIC, | |
| 131 | "TaxState" NUMERIC, | |
| 132 | PRIMARY KEY("PaycheckID" AUTOINCREMENT) | |
| 133 | ) | |
| 134 | ``` | |
| 135 | ||
| 136 | ### Python Reporting | |
| 137 | ||
| 138 | Once I created the database tables and imported all my data, the only step left | |
| 139 | was to create a process to report and visualize on various aspects of the data. | |
| 140 | ||
| 141 | In order to explore and create the reports I'm interested in, I utilized a | |
| 142 | two-part process involving Jupyter Notebooks and Python scripts. | |
| 143 | ||
| 144 | #### Step 1: Jupyter Notebooks | |
| 145 | ||
| 146 | When I need to explore data, try different things, and re-run my code | |
| 147 | cell-by-cell, I use Jupyter Notebooks. For example, I explored the `Accounts` | |
| 148 | table until I found the following useful information: | |
| 149 | ||
| 150 | ```python | |
| 151 | import sqlite3 | |
| 152 | import pandas as pd | |
| 153 | import matplotlib | |
| 154 | ||
| 155 | # Set up database filename and connect | |
| 156 | db = "finances.sqlite" | |
| 157 | connection = sqlite3.connect(db) | |
| 158 | df = pd.read_sql_query("SELECT * FROM Accounts", connection) | |
| 159 | ||
| 160 | # Set global matplotlib variables | |
| 161 | %matplotlib inline | |
| 162 | matplotlib.rcParams['text.color'] = 'white' | |
| 163 | matplotlib.rcParams['axes.labelcolor'] = 'white' | |
| 164 | matplotlib.rcParams['xtick.color'] = 'white' | |
| 165 | matplotlib.rcParams['ytick.color'] = 'white' | |
| 166 | matplotlib.rcParams['legend.labelcolor'] = 'black' | |
| 167 | ||
| 168 | # Display graph | |
| 169 | df.groupby(['AccountType']).sum().plot.pie(title='Credit Line by Account Type', y='CreditLine', figsize=(5,5), autopct='%1.1f%%') | |
| 170 | ``` | |
| 171 | ||
| 172 | #### Step 2: Python Scripts | |
| 173 | ||
| 174 | Once I explored enough through the notebooks and had a list of reports I wanted, | |
| 175 | I moved on to create a Python project with the following structure: | |
| 176 | ||
| 177 | ``` | |
| 178 | finance/ | |
| 179 | ├── notebooks/ | |
| 180 | │ │ ├── account_summary.ipynb | |
| 181 | │ │ ├── account_details.ipynb | |
| 182 | │ │ └── payroll.ipynb | |
| 183 | ├── public/ | |
| 184 | │ │ ├── image-01.png | |
| 185 | │ │ └── image-0X.png | |
| 186 | ├── src/ | |
| 187 | │ └── finance.sqlite | |
| 188 | ├── venv/ | |
| 189 | ├── _init.py | |
| 190 | ├── database.py | |
| 191 | ├── process.py | |
| 192 | ├── requirements.txt | |
| 193 | └── README.md | |
| 194 | ``` | |
| 195 | ||
| 196 | This structure allows me to: | |
| 197 | ||
| 198 | 1. Compile all required python packages into `requirements.txt` for easy | |
| 199 | installation if I move to a new machine. | |
| 200 | 2. Activate a virtual environment in `venv/` so I don't need to maintain a | |
| 201 | system-wide Python environment just for this project. | |
| 202 | 3. Keep my `notebooks/` folder to continuously explore the data as I see fit. | |
| 203 | 4. Maintain a local copy of the database in `src/` for easy access. | |
| 204 | 5. Export reports, images, HTML files, etc. to `public/`. | |
| 205 | ||
| 206 | Now, onto the differences between the code in a Jupyter Notebook and the actual | |
| 207 | Python files. To create the report in the Notebook snippet above, I created the | |
| 208 | following function inside `process.py`: | |
| 209 | ||
| 210 | ```python | |
| 211 | # Create summary pie chart | |
| 212 | def summary_data(accounts: pandas.DataFrame) -> None: | |
| 213 | accounts_01 = accounts[accounts["Owner"] == "Person01"] | |
| 214 | accounts_02 = accounts[accounts["Owner"] == "Person02"] | |
| 215 | for x in range(1, 4): | |
| 216 | if x == 1: | |
| 217 | df = accounts | |
| 218 | account_string = "All Accounts" | |
| 219 | elif x == 2: | |
| 220 | df = accounts_01 | |
| 221 | account_string = "Person01's Accounts" | |
| 222 | elif x == 3: | |
| 223 | df = accounts_02 | |
| 224 | account_string = "Person02's Accounts" | |
| 225 | print(f"Generating pie chart summary image for {account_string}...") | |
| 226 | summary_chart = ( | |
| 227 | df.groupby(["AccountType"]) | |
| 228 | .sum() | |
| 229 | .plot.pie( | |
| 230 | title=f"Credit Line by Type for {account_string}", | |
| 231 | y="CreditLine", | |
| 232 | autopct="%1.1f%%", | |
| 233 | ) | |
| 234 | ) | |
| 235 | summary_chart.figure.savefig(f"public/summary_chart_{x}.png", dpi=1200) | |
| 236 | ``` | |
| 237 | ||
| 238 | The result? A high-quality pie chart that is read directly by the | |
| 239 | `public/index.html` template I use. | |
| 240 | ||
| 241 |  | |
| 242 | ||
| 243 | Other charts generated by this project include: | |
| 244 | ||
| 245 | - Charts of account balances over time. | |
| 246 | - Line chart of effective tax rate (taxes divided by taxable income). | |
| 247 | - Salary projections and error limits using past income and inflation rates. | |
| 248 | - Multi-line chart of gross income, taxable income, and net income. | |
| 249 | ||
| 250 | The best thing about this project? I can improve it at any given time, shaping | |
| 251 | it into whatever helps me the most for that time. I imagine that I will be | |
| 252 | introducing an asset tracking table soon to track the depreciating value of | |
| 253 | cars, houses, etc. Who knows what's next? | |
blog/2022-03-08-plex-migration.org added +233
| @@ -0,0 +1,233 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-03-08 | |
| 3 | title = "Migrating Plex to New Server (+ Nvidia Transcoding)" | |
| 4 | description = "A quick instructional guide on how to migrate Plex Media Server to a new server, including servers with Nvidia GPUs." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Migration Phases | |
| 9 | ||
| 10 | I recently decided to migrate my server from an old OptiPlex desktop machine to | |
| 11 | a custom-built tower with better hardware in every category. In order to do | |
| 12 | this, I would need to properly migrate a full Plex installation. | |
| 13 | ||
| 14 | The second part of this migration is that the new server uses an Nvidia GPU and | |
| 15 | does not have any integrated graphics, which requires extra work for | |
| 16 | installation, but provides much better hardware transcoding options for Plex. | |
| 17 | ||
| 18 | Therefore, I have broken this migration down into three phases: | |
| 19 | ||
| 20 | 1. [Configure the New Server](#phase-1-configure-the-new-server) | |
| 21 | 2. [Migrate Plex Data & Devices](#phase-2-migrate-plex-data-devices) | |
| 22 | 3. [Configure GPU Transcoding](#phase-3-configure-gpu-transcoding) | |
| 23 | ||
| 24 | --- | |
| 25 | ||
| 26 | ## Phase 1: Configure the New Server | |
| 27 | ||
| 28 | ### Choosing an OS | |
| 29 | ||
| 30 | In order to migrate Plex to my new server, I first needed to choose an | |
| 31 | appropriate operating system (OS) and install it on the machine. Given that I | |
| 32 | have encountered numerous issues installing other Linux distributions | |
| 33 | properly with Nvidia graphics, I chose | |
| 34 | [Ubuntu Server](https://ubuntu.com/download/server). | |
| 35 | ||
| 36 | The first step is to create a bootable USB with Ubuntu Server. This is easy with | |
| 37 | [Etcher](https://www.balena.io/etcher/), an app that runs on many different | |
| 38 | platforms. Just download the Ubuntu Server `.iso` image, launch Etcher, and | |
| 39 | install the `.iso` on the USB. | |
| 40 | ||
| 41 | Once the USB is created, insert it into my server, reboot, and click `Esc` (or | |
| 42 | any of the `F1-12` keys) until the BIOS menu appears. Finally, launch the USB | |
| 43 | boot drive. | |
| 44 | ||
| 45 | ### Booting with Nvidia | |
| 46 | ||
| 47 | In order to install Ubuntu Server with an Nvidia Graphics card (and no | |
| 48 | integrated graphics on this device for some reason), you'll have to configure | |
| 49 | the boot menu to allow different graphics drivers to be loaded. | |
| 50 | ||
| 51 | When booting from the USB, the machine will launch the initial installation | |
| 52 | menu. From this menu, type `e` to view the default command options that come | |
| 53 | with the device - it's a good idea to take a photo of this screen, so you can | |
| 54 | enter these commands on the next screen (along with adding support for Nvidia). | |
| 55 | ||
| 56 | Finally, type `Ctrl + C` to enter the command line. From this command line, | |
| 57 | enter the commands found on the `e` screen. **Remember to add `nomodeset` to the | |
| 58 | `linux ...` line so that your Nvidia device will display the installation | |
| 59 | screens properly!** | |
| 60 | ||
| 61 | Here's an example of the commands I pulled from the `e` screen and entered on | |
| 62 | the command line. | |
| 63 | ||
| 64 | ```sh | |
| 65 | setparams 'Install Ubuntu Server' | |
| 66 | setgfxpayload=keep | |
| 67 | linux /casper/vmlinuz quiet nomodeset --- | |
| 68 | initrd /casper/initrd | |
| 69 | boot | |
| 70 | ``` | |
| 71 | ||
| 72 |  | |
| 73 | ||
| 74 |  | |
| 75 | ||
| 76 | Once the machine is rebooted, enter the `e` screen again and add `nomodeset` to | |
| 77 | the `linux ...` line again and press `Ctrl + X` to save the boot options. | |
| 78 | ||
| 79 | The machine is now fully installed and can properly display on an external | |
| 80 | display using the Nvidia GPU. | |
| 81 | ||
| 82 | Always remember to update and upgrade on a new installation: | |
| 83 | ||
| 84 | ```sh | |
| 85 | sudo apt update; sudo apt upgrade -y; sudo apt autoremove -y | |
| 86 | ``` | |
| 87 | ||
| 88 | --- | |
| 89 | ||
| 90 | ## Phase 2: Migrate Plex Data & Devices | |
| 91 | ||
| 92 | This phase uses the great Plex article on migrations | |
| 93 | ([Move an Installation to Another System](https://support.plex.tv/articles/201370363-move-an-install-to-another-system/)) | |
| 94 | and adds a bit more information to help with commands and context. | |
| 95 | ||
| 96 | ### Terminology | |
| 97 | ||
| 98 | **Source:** The original server that is being replaced. | |
| 99 | **Destination:** The new server. | |
| 100 | **Client:** Any application that can be used to modify settings for both | |
| 101 | source/destination. | |
| 102 | ||
| 103 | ### Step 01: [Client] Update Settings | |
| 104 | ||
| 105 | Open up a Plex app and _disable_ the `Account` > `Library` > | |
| 106 | `Empty trash automatically after every scan` preference for the source server. | |
| 107 | ||
| 108 | ### Step 02: [Destination] Install Plex | |
| 109 | ||
| 110 | Open up the | |
| 111 | [Plex Media Server download page](https://www.plex.tv/media-server-downloads/) | |
| 112 | and copy the link for the appropriate platform. | |
| 113 | ||
| 114 | Execute the following commands on the destination server to install Plex: | |
| 115 | ||
| 116 | ```sh | |
| 117 | wget <url> | |
| 118 | sudo dpkg -i <filename> | |
| 119 | sudo systemctl stop plexmediaserver.service | |
| 120 | ``` | |
| 121 | ||
| 122 | ### Step 03: [Source] Stop Plex & Migrate Data | |
| 123 | ||
| 124 | First, stop the Plex service so that no data is created or modified during the | |
| 125 | migration. | |
| 126 | ||
| 127 | ```sh | |
| 128 | sudo systemctl stop plexmediaserver.service | |
| 129 | ``` | |
| 130 | ||
| 131 | Next, copy the data to the new server. To find where the Plex data directory is | |
| 132 | located, Plex has another excellent article available: | |
| 133 | [Where is the Plex Media Server data directory located?](https://support.plex.tv/articles/202915258-where-is-the-plex-media-server-data-directory-located/). | |
| 134 | ||
| 135 | There are many ways to copy the data to the new server and will largely depend | |
| 136 | on the size of the folder being copied. Personally, my data folder was ~23GB and | |
| 137 | I opted to simply use the `scp` command to copy the files over SSH. | |
| 138 | ||
| 139 | This process was throttled by the old server's slow HDD and ports and took | |
| 140 | approximately 90 minutes to complete. In comparison, moving the data from the | |
| 141 | new server's `home/user/` directory to the `/var/.../Plex Media Server` | |
| 142 | directory took 2-3 minutes. | |
| 143 | ||
| 144 | ```sh | |
| 145 | scp -r "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server" your_user@xxx.xxx.xxx.xxx:"'/path/to/destination/'" | |
| 146 | ``` | |
| 147 | ||
| 148 | ### Step 04: [Destination] Update File Permissions | |
| 149 | ||
| 150 | In case you move the data directory to a common area on the new server, it will | |
| 151 | have to be moved to the proper location before Plex can function properly: | |
| 152 | ||
| 153 | ```sh | |
| 154 | mv "Plex Media Server" /var/lib/plexmediaserver/Library/Application Support/ | |
| 155 | ``` | |
| 156 | ||
| 157 | To ensure permissions were retained properly, the server will need to show that | |
| 158 | all files and folders in the data directory are owned by `plex:plex` (or | |
| 159 | whichever user is running the Plex application). | |
| 160 | ||
| 161 | ```sh | |
| 162 | sudo chown -R plex:plex "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server" | |
| 163 | ``` | |
| 164 | ||
| 165 | Finally, start the service and check the status. | |
| 166 | ||
| 167 | ```sh | |
| 168 | sudo systemctl start plexmediaserver.service | |
| 169 | sudo systemctl status plexmediaserver.service | |
| 170 | ``` | |
| 171 | ||
| 172 | ### Step 05: [Client] Update Libraries & Metadata | |
| 173 | ||
| 174 | The first step - now that the new server is up and running - is to sign out of | |
| 175 | the client and sign back in. Once this is done, update any library locations, if | |
| 176 | necessary. This was unnecessary in my case since I simply moved my storage | |
| 177 | drives from the source server to the destination server. | |
| 178 | ||
| 179 | Next, perform the following actions in the client: | |
| 180 | ||
| 181 | 1. On the left sidebar, click `More` > Three-Dot Menu > `Scan Library Files` | |
| 182 | 2. _Enable_ the `Account` > `Library` > | |
| 183 | `Empty trash automatically after every scan` preference for the source | |
| 184 | server. | |
| 185 | 3. On the left sidebar, click `More` > Three-Dot Menu > `Manage Server` > | |
| 186 | `Empty Trash` | |
| 187 | 4. On the left sidebar, click `More` > Three-Dot Menu > `Manage Server` > | |
| 188 | `Clean Bundles` | |
| 189 | 5. On the left sidebar, click `More` > Three-Dot Menu > `Manage Server` > | |
| 190 | `Optimize Database` | |
| 191 | ||
| 192 | Finally, double-check the Remote Access settings to make sure no changes have | |
| 193 | caused issues with accessing the server from outside the network. | |
| 194 | ||
| 195 | In my case, I use a single port forwarding rule in my router and needed to | |
| 196 | update the Local LAN IP Address to the new server IP address. | |
| 197 | ||
| 198 |  | |
| 199 | ||
| 200 | --- | |
| 201 | ||
| 202 | ## Phase 3: Configure GPU Transcoding | |
| 203 | ||
| 204 | The final piece to the migration is enabling hardware transcoding so that Plex | |
| 205 | can fully utilize the new Nvidia GPU available in the server. The first step is | |
| 206 | to install Nvidia graphics drivers. This process may take a few minutes, but the | |
| 207 | commands are pretty simple: | |
| 208 | ||
| 209 | ```sh | |
| 210 | sudo add-apt-repository ppa:graphics-drivers/ppa | |
| 211 | sudo apt update | |
| 212 | sudo apt-get install ubuntu-drivers-common | |
| 213 | sudo ubuntu-drivers autoinstall | |
| 214 | ``` | |
| 215 | ||
| 216 | Finally, reboot so that the changes are loaded: | |
| 217 | ||
| 218 | ```sh | |
| 219 | sudo reboot now | |
| 220 | ``` | |
| 221 | ||
| 222 | To ensure that the Nvidia graphics drivers are working properly, run the | |
| 223 | following command to view the available GPUs, statistics, and processes: | |
| 224 | ||
| 225 | ```sh | |
| 226 | sudo nvidia-smi | |
| 227 | ``` | |
| 228 | ||
| 229 |  | |
| 230 | ||
| 231 | Finally, enable hardware transcoding settings in the Plex application: | |
| 232 | ||
| 233 |  | |
blog/2022-03-23-cloudflare-dns-api.org added +190
| @@ -0,0 +1,190 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-03-23 | |
| 3 | title = "Dynamic DNS with Cloudflare API" | |
| 4 | description = "Learn how to update Cloudflare DNS records automatically with a simple bash script." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## DDNS: Dynamic DNS | |
| 9 | ||
| 10 | If you're hosting a service from a location with dynamic DNS (where your IP may | |
| 11 | change at any time), you must have a solution to update the DNS so that you can | |
| 12 | access your service even when the IP of the server changes. | |
| 13 | ||
| 14 | The process below uses the [Cloudflare API](https://api.cloudflare.com/) to | |
| 15 | update DNS `A` and `AAAA` records with the server's current IP. If you use | |
| 16 | another DNS provider, you will have to find a way to update your DNS (or find a | |
| 17 | way to get a static IP). | |
| 18 | ||
| 19 | First, install `jq` since we will use it in the next script: | |
| 20 | ||
| 21 | ```sh | |
| 22 | sudo apt install jq | |
| 23 | ``` | |
| 24 | ||
| 25 | Next, create a location for your DDNS update scripts and open the first script: | |
| 26 | ||
| 27 | ```sh | |
| 28 | mkdir ~/ddns | |
| 29 | nano ~/ddns/update.sh | |
| 30 | ``` | |
| 31 | ||
| 32 | The following `update.sh` script will take all of your domains and subdomains | |
| 33 | and check Cloudflare to see if the current `A` and `AAAA` records match your | |
| 34 | server's IP address. If not, it will update the records. | |
| 35 | ||
| 36 | ```sh | |
| 37 | # file: update.sh | |
| 38 | #!/bin/bash | |
| 39 | ||
| 40 | # Update TLDs | |
| 41 | domains=(example.com example.net) | |
| 42 | ||
| 43 | for domain in "${domains[@]}" | |
| 44 | do | |
| 45 | echo -e "\nUpdating $domain..." | |
| 46 | zone_name=$domain /home/<your-username>/ddns/ddns.sh | |
| 47 | done | |
| 48 | ||
| 49 | # Update subdomains for example.com | |
| 50 | domain=example.com | |
| 51 | subdomains=(photos.example.com) | |
| 52 | ||
| 53 | for subdomain in "${subdomains[@]}" | |
| 54 | do | |
| 55 | echo -e "\nUpdating $subdomain..." | |
| 56 | zone_name=$domain dns_record=$subdomain /home/<your-username>/ddns/ddns.sh | |
| 57 | done | |
| 58 | ``` | |
| 59 | ||
| 60 | Next, open up the `ddns.sh` script. Paste the following into the script and | |
| 61 | update the `api_token` and `email` variables. | |
| 62 | ||
| 63 | ```sh | |
| 64 | nano ~/ddns/ddns.sh | |
| 65 | ``` | |
| 66 | ||
| 67 | :warning: **Note**: If you want your DNS records to be proxied through | |
| 68 | Cloudflare, find and update the following snippet: `\"proxied\":false}"` to say | |
| 69 | `true` instead of `false`. | |
| 70 | ||
| 71 | ```sh | |
| 72 | # file: ddns.sh | |
| 73 | #!/bin/bash | |
| 74 | # based on https://gist.github.com/Tras2/cba88201b17d765ec065ccbedfb16d9a | |
| 75 | # initial data; they need to be filled by the user | |
| 76 | ## API token | |
| 77 | api_token=<YOUR_API_TOKEN> | |
| 78 | ## email address associated with the Cloudflare account | |
| 79 | email=<YOUR_EMAIL> | |
| 80 | ||
| 81 | # get the basic data | |
| 82 | ipv4=$(curl -s -X GET -4 https://ifconfig.co) | |
| 83 | ipv6=$(curl -s -X GET -6 https://ifconfig.co) | |
| 84 | user_id=$(curl -s -X GET "https://api.cloudflare.com/client/v4/user/tokens/verify" \ | |
| 85 | -H "Authorization: Bearer $api_token" \ | |
| 86 | -H "Content-Type:application/json" \ | |
| 87 | | jq -r '{"result"}[] | .id' | |
| 88 | ) | |
| 89 | ||
| 90 | echo "Your IPv4 is: $ipv4" | |
| 91 | echo "Your IPv6 is: $ipv6" | |
| 92 | ||
| 93 | # check if the user API is valid and the email is correct | |
| 94 | if [ $user_id ] | |
| 95 | then | |
| 96 | zone_id=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones?name=$zone_name&status=active" \ | |
| 97 | -H "Content-Type: application/json" \ | |
| 98 | -H "X-Auth-Email: $email" \ | |
| 99 | -H "Authorization: Bearer $api_token" \ | |
| 100 | | jq -r '{"result"}[] | .[0] | .id' | |
| 101 | ) | |
| 102 | # check if the zone ID is | |
| 103 | if [ $zone_id ] | |
| 104 | then | |
| 105 | # check if there is any IP version 4 | |
| 106 | if [ $ipv4 ] | |
| 107 | then | |
| 108 | dns_record_a_id=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$zone_id/dns_records?type=A&name=$dns_record" \ | |
| 109 | -H "Content-Type: application/json" \ | |
| 110 | -H "X-Auth-Email: $email" \ | |
| 111 | -H "Authorization: Bearer $api_token" | |
| 112 | ) | |
| 113 | # if the IPv6 exist | |
| 114 | dns_record_a_ip=$(echo $dns_record_a_id | jq -r '{"result"}[] | .[0] | .content') | |
| 115 | echo "The set IPv4 on Cloudflare (A Record) is: $dns_record_a_ip" | |
| 116 | if [ $dns_record_a_ip != $ipv4 ] | |
| 117 | then | |
| 118 | # change the A record | |
| 119 | curl -s -X PUT "https://api.cloudflare.com/client/v4/zones/$zone_id/dns_records/$(echo $dns_record_a_id | jq -r '{"result"}[] | .[0] | .id')" \ | |
| 120 | -H "Content-Type: application/json" \ | |
| 121 | -H "X-Auth-Email: $email" \ | |
| 122 | -H "Authorization: Bearer $api_token" \ | |
| 123 | --data "{\"type\":\"A\",\"name\":\"$dns_record\",\"content\":\"$ipv4\",\"ttl\":1,\"proxied\":false}" \ | |
| 124 | | jq -r '.errors' | |
| 125 | else | |
| 126 | echo "The current IPv4 and DNS record IPv4 are the same." | |
| 127 | fi | |
| 128 | else | |
| 129 | echo "Could not get your IPv4. Check if you have it; e.g. on https://ifconfig.co" | |
| 130 | fi | |
| 131 | ||
| 132 | # check if there is any IP version 6 | |
| 133 | if [ $ipv6 ] | |
| 134 | then | |
| 135 | dns_record_aaaa_id=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$zone_id/dns_records?type=AAAA&name=$dns_record" \ | |
| 136 | -H "Content-Type: application/json" \ | |
| 137 | -H "X-Auth-Email: $email" \ | |
| 138 | -H "Authorization: Bearer $api_token" | |
| 139 | ) | |
| 140 | # if the IPv6 exist | |
| 141 | dns_record_aaaa_ip=$(echo $dns_record_aaaa_id | jq -r '{"result"}[] | .[0] | .content') | |
| 142 | echo "The set IPv6 on Cloudflare (AAAA Record) is: $dns_record_aaaa_ip" | |
| 143 | if [ $dns_record_aaaa_ip != $ipv6 ] | |
| 144 | then | |
| 145 | # change the AAAA record | |
| 146 | curl -s -X PUT "https://api.cloudflare.com/client/v4/zones/$zone_id/dns_records/$(echo $dns_record_aaaa_id | jq -r '{"result"}[] | .[0] | .id')" \ | |
| 147 | -H "Content-Type: application/json" \ | |
| 148 | -H "X-Auth-Email: $email" \ | |
| 149 | -H "Authorization: Bearer $api_token" \ | |
| 150 | --data "{\"type\":\"AAAA\",\"name\":\"$dns_record\",\"content\":\"$ipv6\",\"ttl\":1,\"proxied\":false}" \ | |
| 151 | | jq -r '.errors' | |
| 152 | else | |
| 153 | echo "The current IPv6 and DNS record IPv6 are the same." | |
| 154 | fi | |
| 155 | else | |
| 156 | echo "Could not get your IPv6. Check if you have it; e.g. on https://ifconfig.co" | |
| 157 | fi | |
| 158 | else | |
| 159 | echo "There is a problem with getting the Zone ID. Check if the Zone Name is correct." | |
| 160 | fi | |
| 161 | else | |
| 162 | echo "There is a problem with either the email or the password" | |
| 163 | fi | |
| 164 | ``` | |
| 165 | ||
| 166 | Once the script is saved and closed, make the scripts executable: | |
| 167 | ||
| 168 | ```sh | |
| 169 | chmod +x ~/ddns/ddns.sh | |
| 170 | chmod +x ~/ddns/update.sh | |
| 171 | ``` | |
| 172 | ||
| 173 | You can test the script by running it manually: | |
| 174 | ||
| 175 | ```sh | |
| 176 | ./update.sh | |
| 177 | ``` | |
| 178 | ||
| 179 | To make sure the scripts run automatically, add it to the `cron` file so that it | |
| 180 | will run on a schedule. To do this, open the cron file: | |
| 181 | ||
| 182 | ```sh | |
| 183 | crontab -e | |
| 184 | ``` | |
| 185 | ||
| 186 | In the cron file, paste the following at the bottom of the editor: | |
| 187 | ||
| 188 | ```sh | |
| 189 | */5 * * * * bash /home/<your_username>/ddns/update.sh | |
| 190 | ``` | |
blog/2022-03-23-nextcloud-on-ubuntu.org added +161
| @@ -0,0 +1,161 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-03-23 | |
| 3 | title = "Installing Nextcloud on Ubuntu" | |
| 4 | description = "Learn how to install the self-hosted cloud storage solution, Nextcloud, on Ubuntu." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## What is Nextcloud? | |
| 9 | ||
| 10 | [Nextcloud](https://nextcloud.com/) is a self-hosted solution for storage, | |
| 11 | communications, editing, calendar, contacts, and more. | |
| 12 | ||
| 13 | This tutorial assumes that you have an Ubuntu server and a domain name | |
| 14 | configured to point toward the server. | |
| 15 | ||
| 16 | ## Install Dependencies | |
| 17 | ||
| 18 | To start, you will need to install the packages that Nextcloud requires: | |
| 19 | ||
| 20 | ```sh | |
| 21 | sudo apt install apache2 mariadb-server libapache2-mod-php7.4 | |
| 22 | sudo apt install php7.4-gd php7.4-mysql php7.4-curl php7.4-mbstring php7.4-intl | |
| 23 | sudo apt install php7.4-gmp php7.4-bcmath php-imagick php7.4-xml php7.4-zip | |
| 24 | ``` | |
| 25 | ||
| 26 | ## Set Up MySQL | |
| 27 | ||
| 28 | Next, you will need to log in to MySQL as the `root` user of the machine. | |
| 29 | ||
| 30 | ```sh | |
| 31 | sudo mysql -uroot -p | |
| 32 | ``` | |
| 33 | ||
| 34 | Once you've logged in, you must create a new user so that Nextcloud can manage | |
| 35 | the database. You will also create a `nextcloud` database and assign privileges: | |
| 36 | ||
| 37 | ```sql | |
| 38 | CREATE USER 'username'@'localhost' IDENTIFIED BY 'password'; | |
| 39 | CREATE DATABASE IF NOT EXISTS nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; | |
| 40 | GRANT ALL PRIVILEGES ON nextcloud.* TO 'username'@'localhost'; | |
| 41 | FLUSH PRIVILEGES; | |
| 42 | quit; | |
| 43 | ``` | |
| 44 | ||
| 45 | ## Download & Install Nextcloud | |
| 46 | ||
| 47 | To download Nextcloud, go the | |
| 48 | [Nextcloud downloads page](https://nextcloud.com/install/#instructions-server), | |
| 49 | click on `Archive File` and right-click the big blue button to copy the link. | |
| 50 | ||
| 51 | Then, go to your server and enter the following commands to download, unzip, and | |
| 52 | move the files to your destination directory. This example uses `example.com` as | |
| 53 | the destination, but you can put it wherever you want to server your files from. | |
| 54 | ||
| 55 | ```sh | |
| 56 | wget https://download.nextcloud.com/server/releases/nextcloud-23.0.3.zip | |
| 57 | sudo apt install unzip | |
| 58 | unzip nextcloud-23.0.3.zip | |
| 59 | sudo cp -r nextcloud /var/www/example.com | |
| 60 | ``` | |
| 61 | ||
| 62 | ## Configure the Apache Web Server | |
| 63 | ||
| 64 | Now that the database is set up and Nextcloud is installed, you need to set up | |
| 65 | the Apache configuration files to tell the server how to handle requests for | |
| 66 | `example.com/nextcloud`. | |
| 67 | ||
| 68 | First, open the following file in the editor: | |
| 69 | ||
| 70 | ```sh | |
| 71 | sudo nano /etc/apache2/sites-available/nextcloud.conf | |
| 72 | ``` | |
| 73 | ||
| 74 | Once the editor is open, paste the following information in. Then, save and | |
| 75 | close the file. | |
| 76 | ||
| 77 | ```config | |
| 78 | <VirtualHost *:80> | |
| 79 | DocumentRoot /var/www/example.com | |
| 80 | ServerName example.com | |
| 81 | ServerAlias www.example.com | |
| 82 | ErrorLog ${APACHE_LOG_DIR}/error.log | |
| 83 | CustomLog ${APACHE_LOG_DIR}/access.log combined | |
| 84 | ||
| 85 | <Directory /var/www/example.com/nextcloud/> | |
| 86 | Require all granted | |
| 87 | AllowOverride All | |
| 88 | Options FollowSymLinks MultiViews | |
| 89 | Satisfy Any | |
| 90 | ||
| 91 | <IfModule mod_dav.c> | |
| 92 | Dav off | |
| 93 | </IfModule> | |
| 94 | </Directory> | |
| 95 | </VirtualHost> | |
| 96 | ``` | |
| 97 | ||
| 98 | Once the file is saved, enable it with Apache: | |
| 99 | ||
| 100 | ```sh | |
| 101 | sudo a2ensite nextcloud.conf | |
| 102 | ``` | |
| 103 | ||
| 104 | Next, enable the Apache mods required by Nextcloud: | |
| 105 | ||
| 106 | ```sh | |
| 107 | sudo a2enmod rewrite headers env dir mime | |
| 108 | ``` | |
| 109 | ||
| 110 | Finally, restart Apache. If any errors arise, you must solve those before | |
| 111 | continuing. | |
| 112 | ||
| 113 | ```sh | |
| 114 | sudo systemctl restart apache2 | |
| 115 | ``` | |
| 116 | ||
| 117 | For the app to work, you must have the correct file permissions on your | |
| 118 | `nextcloud` directory. Set the owner to be `www-data`: | |
| 119 | ||
| 120 | ```sh | |
| 121 | sudo chown -R www-data:www-data /var/www/example.com/nextcloud/ | |
| 122 | ``` | |
| 123 | ||
| 124 | ## DNS | |
| 125 | ||
| 126 | If you do not have a static IP address, you will need to update your DNS | |
| 127 | settings (at your DNS provider) whenever your dynamic IP address changes. | |
| 128 | ||
| 129 | For an example on how I do that with Cloudflare, see my other post: | |
| 130 | [Updating Dynamic DNS with Cloudflare API](/blog/updating-dynamic-dns-with-cloudflare-api/) | |
| 131 | ||
| 132 | ## Certbot | |
| 133 | ||
| 134 | If you want to serve Nextcloud from HTTPS rather than plain HTTP, use the | |
| 135 | following commands to issue Let's Encrypt SSL certificates: | |
| 136 | ||
| 137 | ```sh | |
| 138 | sudo apt install snapd | |
| 139 | sudo snap install core | |
| 140 | sudo snap refresh core | |
| 141 | sudo snap install --classic certbot | |
| 142 | sudo ln -s /snap/bin/certbot /usr/bin/certbot | |
| 143 | sudo certbot --apache | |
| 144 | ``` | |
| 145 | ||
| 146 | ## Results | |
| 147 | ||
| 148 | Voilà! | |
| 149 | You're all done and should be able to access Nextcloud from your domain or | |
| 150 | IP address. | |
| 151 | ||
| 152 | See the screenshots below for the dashboard and a settings page on my instance | |
| 153 | of Nextcloud, using the `Breeze Dark` theme I installed from the Apps page. | |
| 154 | ||
| 155 |  | |
| 156 | ||
| 157 | _Figure 01: Nextcloud Dashboard_ | |
| 158 | ||
| 159 |  | |
| 160 | ||
| 161 | _Figure 02: Nextcloud Security Settings_ | |
blog/2022-03-24-server-hardening.org added +346
| @@ -0,0 +1,346 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-03-24 | |
| 3 | title = "Hardening a Public-Facing Home Server" | |
| 4 | description = "My personal reflection on the steps it took to get a home server ready to be publicly-accessible." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Post Updates | |
| 9 | ||
| 10 | > After reviewing this post today (2022-10-04), I noticed quite a few gaps in my | |
| 11 | > write-up and wanted to add a few things, even though this blog is really | |
| 12 | > just a retrospective and knowledge dump for myself. | |
| 13 | > I left things intact and simply crossed them out (~~like this~~) for | |
| 14 | > posterity. | |
| 15 | ||
| 16 | ## Planning Data Flows & Security | |
| 17 | ||
| 18 | ### My Personal Data Flow | |
| 19 | ||
| 20 | ```txt | |
| 21 | ┌───────┐ ┌─────────────────┐ | |
| 22 | ┌──► VLAN1 ├───► Private Devices │ | |
| 23 | │ └───────┘ └─────────────────┘ | |
| 24 | ┌──────────┐ ┌────────┐ ┌──────────┐ ┌────────┐ │ | |
| 25 | │ Internet ├───► Router ├───► Firewall ├───► Switch ├──┤ | |
| 26 | └──────────┘ └────────┘ └──────────┘ └────────┘ │ | |
| 27 | │ ┌───────┐ ┌───────────────┐ | |
| 28 | └──► VLAN2 ├───► Public Server │ | |
| 29 | └───────┘ └───────────────┘ | |
| 30 | ``` | |
| 31 | ||
| 32 | ### Thought Process | |
| 33 | ||
| 34 | To serve content from your home server and harden your security posture, you | |
| 35 | have to think about the transport of data from `server` to `client`. | |
| 36 | ||
| 37 | Let's start with the actual server itself. Think about the following: | |
| 38 | ||
| 39 | - Do I have a firewall enabled? Do I need to update this to allow new ports or | |
| 40 | IPs? | |
| 41 | - Do I have an IPS/IDS that may prevent outside traffic? | |
| 42 | - Do I have any other security software installed? | |
| 43 | - Are the services hosted inside Docker containers, behind a reverse proxy, or | |
| 44 | virtualized? If so, are they configured to allow outside traffic? | |
| 45 | ||
| 46 | Once the data leaves the server, where does it go? In my case, it goes to a | |
| 47 | managed switch. In this case, I asked the following: | |
| 48 | ||
| 49 | - What configurations is the switch using? | |
| 50 | - Am I using VLANs? | |
| 51 | - Yes, I am using 802.1Q VLANs. | |
| 52 | - Are the VLANs configured properly? | |
| 53 | - Yes, as shown in the [Switch](#switch) section below, I have a separate | |
| 54 | VLAN to allow outside traffic to and from the server alone. No other | |
| 55 | devices, except for a service port, and in that VLAN. | |
| 56 | ||
| 57 | At this point, the data has been processed through the switch. Where does it go | |
| 58 | next? In my case, it's pretty simple: it goes to the router/modem device. | |
| 59 | ||
| 60 | - Does my ISP block any ports that I need? | |
| 61 | - This is an important step that a lot of people run into when | |
| 62 | self-hosting at home. | |
| 63 | Use an online port-checker tool for your IP or call your ISP if you | |
| 64 | think ports are blocked. | |
| 65 | - Is there a router firewall? | |
| 66 | - Yes, I checked that it's configured to allow the ports I need to run my | |
| 67 | services publicly. Common web servers and reverse proxies require ports | |
| 68 | 80 and 443, but other services like media servers or games can require | |
| 69 | unique ports, so be sure to check the documentation for your service(s). | |
| 70 | - Are there any other settings affecting inbound/outbound traffic? | |
| 71 | - Schedules or access blocks | |
| 72 | - Static Routing | |
| 73 | - QoS | |
| 74 | - Port Forwarding | |
| 75 | - DMZ Hosting | |
| 76 | - Remote Management (this can sometimes mess with services that also | |
| 77 | require the use of ports 80 and 443) | |
| 78 | ||
| 79 | Once the data leaves my router, it goes to the upstream ISP and can be accessed | |
| 80 | publicly. | |
| 81 | ||
| 82 | ## Server | |
| 83 | ||
| 84 | ~~The services I run on my server are installed straight into the OS, without | |
| 85 | any use of Docker or VMs, so I don't need any extra application configuration to | |
| 86 | make them accessible to the outside world.~~ | |
| 87 | ||
| 88 | As of 2022-10-04, the paragraph above is no longer true as I now run a reverse | |
| 89 | proxy with Nginx and host many services inside Docker. However, it doesn't | |
| 90 | change anything regarding this post as I still just need to open ports 80 & 443 | |
| 91 | and create the necessary website configuration files. | |
| 92 | ||
| 93 | When creating new services - either installed directly on bare metal or within | |
| 94 | something like Docker - I ensure that I read through the documentation | |
| 95 | thoroughly to understand a few key things: | |
| 96 | - What network activities should this app perform (if any)? Using which ports | |
| 97 | and protocols? | |
| 98 | - Does this app require any commands/services to be run as `root`? | |
| 99 | - Does this app log errors, authentication failures/successes, or anything | |
| 100 | else that would be useful for an investigation? | |
| 101 | ||
| 102 | For extra security, I use limit all incoming connections to SSH connections | |
| 103 | through my server firewall (`ufw`) and disable common SSH settings. After all of | |
| 104 | that, I use `fail2ban` as a preventative measure against brute-force login | |
| 105 | attempts. | |
| 106 | ||
| 107 | As another piece of security, you can randomize your SSH port to ensure that | |
| 108 | random scanners or attackers can't easily try to force their way into your | |
| 109 | network. For example, you can edit the port rules in your server to block all | |
| 110 | connection requests to port `22` but forward all remote connections from port | |
| 111 | `12345` to your server's port `22`. Then you just need to SSH to your network | |
| 112 | via your randomized port. | |
| 113 | ||
| 114 | ### `ufw` | |
| 115 | ||
| 116 | To see how to configure `ufw`, see my other post: | |
| 117 | [Secure Your Network with the Uncomplicated Firewall](/blog/secure-your-network-with-the-uncomplicated-firewall/). | |
| 118 | ||
| 119 | The general notion with an on-device firewall is that you want to deny all | |
| 120 | incoming connections by default and then selectively open certain ports for | |
| 121 | services or users that you know need access. | |
| 122 | ||
| 123 | If you know that you will only be logging into this server from a certain set or | |
| 124 | list of IPs, you can always set the firewall to only allow connections to port | |
| 125 | 22 from those IPs. | |
| 126 | ||
| 127 | For a quick start to only allow SSH connections to the server, use this: | |
| 128 | ||
| 129 | ```sh | |
| 130 | sudo ufw default deny incoming | |
| 131 | sudo ufw default allow outgoing | |
| 132 | sudo ufw allow 22 | |
| 133 | sudo ufw enable | |
| 134 | ``` | |
| 135 | ||
| 136 |  | |
| 137 | ||
| 138 | ### `ssh` | |
| 139 | ||
| 140 | #### Using SSH Keys | |
| 141 | ||
| 142 | First, make sure you have an SSH keypair generated on the device(s) that you'll | |
| 143 | be using to log in to the server. If you don't have an SSH key, run this | |
| 144 | command: | |
| 145 | ||
| 146 | ```sh | |
| 147 | ssh-keygen | |
| 148 | ``` | |
| 149 | ||
| 150 | Now that we have an SSH key, copy it to the server with the following command, | |
| 151 | which will ask for the user's password before accepting the key: | |
| 152 | ||
| 153 | ```sh | |
| 154 | ssh-copy-id my_user@my_server | |
| 155 | ``` | |
| 156 | ||
| 157 | If you have multiple keys, you'll need to specify which to use. After it's | |
| 158 | complete, `ssh` back into the server as that user and make sure it doesn't ask | |
| 159 | for a password. | |
| 160 | ||
| 161 | #### Disable Password & Root Authentication | |
| 162 | ||
| 163 | Now that we can access the server without a password, we will disable password | |
| 164 | authentication and disable anyone from using `ssh` to login as `root`. | |
| 165 | ||
| 166 | To do this, open the `sshd_config` file: | |
| 167 | ||
| 168 | ```sh | |
| 169 | sudo nano /etc/ssh/sshd_config | |
| 170 | ``` | |
| 171 | ||
| 172 | You'll need to update the parameters to the values below. If one of these rules | |
| 173 | is commented-out or doesn't exist, create the rule at the bottom of the file. | |
| 174 | ||
| 175 | ```config | |
| 176 | PermitRootLogin no | |
| 177 | PasswordAuthentication no | |
| 178 | PubkeyAuthentication yes | |
| 179 | ``` | |
| 180 | ||
| 181 | Finally, restart the `ssh` service: | |
| 182 | ||
| 183 | ```sh | |
| 184 | sudo systemctl restart sshd.service | |
| 185 | ``` | |
| 186 | ||
| 187 | To test that everything’s working so far, open ANOTHER terminal and try logging | |
| 188 | in as `root` over SSH. It is very important that you keep your current SSH | |
| 189 | session open and test with an additional session, or you will lock yourself out | |
| 190 | at some point and will need to use a recovery method (e.g., hooking monitor up | |
| 191 | to home server) to get yourself back in. | |
| 192 | ||
| 193 | #### Enable MFA for `ssh` | |
| 194 | ||
| 195 | This part is optional, but I highly recommend it. | |
| 196 | So far, we've ensured that no one can log into our user on the server | |
| 197 | without using our secret key, and we've ensured that no one can log in | |
| 198 | remotely as `root`. | |
| 199 | Next, you can enable MFA authentication for `ssh` connections. | |
| 200 | ||
| 201 | This process involves editing a couple files and installing an MFA package, so I | |
| 202 | will not include all the details in this post. To see how to configure MFA for | |
| 203 | `ssh`, see my other post: | |
| 204 | [Enabling MFA for SSH](/blog/enable-totp-mfa-for-ssh/). | |
| 205 | ||
| 206 |  | |
| 207 | ||
| 208 | ### `fail2ban` | |
| 209 | ||
| 210 | I haven't written a post on how I use `fail2ban`, but it's quite simple. I use | |
| 211 | the default `sshd` jail, but you can always create new jails for respective | |
| 212 | applications or ports. For example, if you use Nginx as your web server, you can | |
| 213 | use the `nginx-http-auth` jail. | |
| 214 | ||
| 215 | In order to get it up and running, use the following commands: | |
| 216 | ||
| 217 | ```sh | |
| 218 | sudo apt install fail2ban | |
| 219 | sudo fail2ban-client start sshd | |
| 220 | sudo fail2ban-client status sshd | |
| 221 | ``` | |
| 222 | ||
| 223 | This should be used as a last-resort defense and shouldn't be a replacement for | |
| 224 | the security measures mentioned above. | |
| 225 | ||
| 226 |  | |
| 227 | ||
| 228 | ## Switch | |
| 229 | ||
| 230 | Between the router and any local devices is my managed switch, which is used to | |
| 231 | create VLANs. The example below shows how I would isolate the VLANs if I were | |
| 232 | starting to host a single service at home. | |
| 233 | ||
| 234 | ### 802.1Q VLAN Configuration | |
| 235 | ||
| 236 | In this configuration, port 8 is the public server that needs to be accessed | |
| 237 | from the outside. Port 23 is my 'dedicated service port' for this server. In | |
| 238 | order to SSH to this server, I need to plug my laptop into port 23 or else I | |
| 239 | cannot SSH. Otherwise, I'd need to hook up a monitor and keyboard directly to | |
| 240 | the server to manage it. | |
| 241 | ||
| 242 | | VLAN ID | VLAN Name | Member Ports | Tagged Ports | Untagged Ports | | |
| 243 | |:-------:|:---------:|:------------:|:------------:|:--------------:| | |
| 244 | | 1 | Default | 1-24 | | 1-24 | | |
| 245 | | 2 | Server | 1,8,23 | | 1,8,23 | | |
| 246 | ||
| 247 | ### 802.1Q VLAN PVID Setting | |
| 248 | ||
| 249 | Once the VLAN is created, I simply add the `VLAN ID` of `2` as the `PVID` for | |
| 250 | any related ports (in this case, see that ports `8` and `23` have a PVID of | |
| 251 | `2`). | |
| 252 | ||
| 253 | | Port | PVID | | |
| 254 | |:----:|:----:| | |
| 255 | | 1 | 1 | | |
| 256 | | 2 | 1 | | |
| 257 | | 3 | 1 | | |
| 258 | | 4 | 1 | | |
| 259 | | 5 | 1 | | |
| 260 | | 6 | 1 | | |
| 261 | | 7 | 1 | | |
| 262 | | 8 | 2 | | |
| 263 | | 9 | 1 | | |
| 264 | | 10 | 1 | | |
| 265 | | 11 | 1 | | |
| 266 | | 12 | 1 | | |
| 267 | | 13 | 1 | | |
| 268 | | 14 | 1 | | |
| 269 | | 15 | 1 | | |
| 270 | | 16 | 1 | | |
| 271 | | 17 | 1 | | |
| 272 | | 18 | 1 | | |
| 273 | | 19 | 1 | | |
| 274 | | 20 | 1 | | |
| 275 | | 21 | 1 | | |
| 276 | | 22 | 1 | | |
| 277 | | 23 | 2 | | |
| 278 | | 24 | 1 | | |
| 279 | ||
| 280 | ## Router | |
| 281 | ||
| 282 | On my router, the configuration was as easy as opening the firewall settings and | |
| 283 | unblocking the ports I needed for my services (e.g., HTTP/S, Plex, SSH, MySQL, | |
| 284 | etc.). | |
| 285 | ||
| 286 | ~~Since I'm relying on an ISP-provided modem/router combo for now (not by | |
| 287 | choice), I do not use any other advanced settings on my router that would | |
| 288 | inhibit any valid traffic to these services.~~ | |
| 289 | ||
| 290 | The paragraph above regarding the ISP-owned router is no longer accurate as I | |
| 291 | now use the Ubiquiti Unifi Dream Machine Pro as my router. | |
| 292 | Within this router, I enabled port forwarding/firewall rules, segregate the | |
| 293 | network based on the device, and enable traffic restrictions (e.g., silently | |
| 294 | drop traffic from certain countries and threat categories). | |
| 295 | ||
| 296 | If you have the option with your ISP, I recommend using a personal router with | |
| 297 | software that you are familiar with so that you can explore all the options | |
| 298 | available to you. | |
| 299 | ||
| 300 | ## Physical Security | |
| 301 | ||
| 302 | One large piece of self-hosting that people generally don't discuss online is | |
| 303 | physical security. However, physical security is very important for everyone who | |
| 304 | hosts a server like this. Exactly *how* important it is depends on the server | |
| 305 | use/purpose. | |
| 306 | ||
| 307 | If you self-host customer applications that hold protected data (HIPAA, GDPR, | |
| 308 | COPPA, etc.), then physical security is extremely important and cannot be | |
| 309 | ignored. If you simply host a blog and some hobby sites, then it's a relatively | |
| 310 | minor consideration, but one you still need to think about. | |
| 311 | ||
| 312 | ### Location | |
| 313 | ||
| 314 | The first consideration is quite simple: location. | |
| 315 | - Is the server within a property you own or housed on someone else's property? | |
| 316 | - Is it nearby (in your house, in your work office, in your neighbor's garage, | |
| 317 | in a storage unit, etc.)? | |
| 318 | - Do you have 24/7 access to the server? | |
| 319 | - Are there climate considerations, such as humidity, fires, tornadoes, | |
| 320 | monsoons? | |
| 321 | - Do you have emergency equipment nearby in case of emergency? | |
| 322 | ||
| 323 | ### Hardware Ownership | |
| 324 | ||
| 325 | Secondly, consider the hardware itself: | |
| 326 | - Do you own the server in its entirety? | |
| 327 | - Are any other users able to access the server, even if your data/space is | |
| 328 | segregated? | |
| 329 | - If you're utilizing a third party, do they have any documentation to show | |
| 330 | responsibility? This could be a SOC 1/2/3 report, ISO compliance report, | |
| 331 | internal security/safety documentation. | |
| 332 | ||
| 333 | ### Physical Controls | |
| 334 | ||
| 335 | Regardless of who owns the hardware, ensure that there are adequate safeguards | |
| 336 | in place, if necessary. These usually don't apply to small home servers and are | |
| 337 | usually covered already if you're utilizing a third party. | |
| 338 | ||
| 339 | These can include: | |
| 340 | - Server bezel locks | |
| 341 | - Server room locks - physical, digital, or biometric authentication | |
| 342 | - Security cameras | |
| 343 | - Raised floors/lowered ceilings with proper guards/gates in-place within the | |
| 344 | floors or ceilings | |
| 345 | - Security personnel | |
| 346 | - Log sheets and/or guest badges | |
blog/2022-03-26-ssh-mfa.org added +196
| @@ -0,0 +1,196 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-03-26 | |
| 3 | title = "Enable TOTP MFA for SSH" | |
| 4 | description = "Harden your SSH target with user-enabled MFA options, such as TOTP." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Why Do I Need MFA for SSH? | |
| 9 | ||
| 10 | If you are a sysadmin of a server anywhere (that includes at home!), you may | |
| 11 | want an added layer of protection against intruders. This is not a replacement | |
| 12 | for other security measures, such as: | |
| 13 | ||
| 14 | - Disable root SSH | |
| 15 | - Disable SSH password authentication | |
| 16 | - Allow only certain users to login via SSH | |
| 17 | - Allow SSH only from certain IPs | |
| 18 | ||
| 19 | However, MFA can be added as an additional security measure to ensure that your | |
| 20 | server is protected. This is especially important if you need to allow password | |
| 21 | authentication for SSH. | |
| 22 | ||
| 23 | For more guidance on server security measures, see my other post: | |
| 24 | [Hardening a Public-Facing Home Server](/blog/hardening-a-public-facing-home-server/). | |
| 25 | ||
| 26 | ## Install MFA PAM Module | |
| 27 | ||
| 28 | PAM, which stands for Pluggable Authentication Module, is an authentication | |
| 29 | infrastructure used on Linux systems to authenticate a user. In order to use | |
| 30 | this technology, let's install the `libpam-google-authenticator` package: | |
| 31 | ||
| 32 | ```sh | |
| 33 | sudo apt-get update | |
| 34 | ``` | |
| 35 | ||
| 36 | ```sh | |
| 37 | sudo apt-get install libpam-google-authenticator | |
| 38 | ``` | |
| 39 | ||
| 40 | ## Initialize the PAM Module | |
| 41 | ||
| 42 | ### Interactive Method | |
| 43 | ||
| 44 | Once the package is installed, initialize it and following the interactive | |
| 45 | prompts to generate your OTP or TOTP: | |
| 46 | ||
| 47 | ```sh | |
| 48 | google-authenticator | |
| 49 | ``` | |
| 50 | ||
| 51 | If you are not sure how to answer, read the prompts carefully and think about | |
| 52 | having to how each situation would affect your normal login attempts. If you are | |
| 53 | still not sure, use my default responses below. | |
| 54 | ||
| 55 | ```txt | |
| 56 | OUTPUT | |
| 57 | ||
| 58 | Do you want authentication tokens to be time-based (y/n) y | |
| 59 | ``` | |
| 60 | ||
| 61 | At this point, use an authenticator app somewhere one of your devices to scan | |
| 62 | the QR code. Any future login attempts after our upcoming configuration changes | |
| 63 | will require that TOTP. | |
| 64 | ||
| 65 | ```txt | |
| 66 | OUTPUT | |
| 67 | ||
| 68 | Do you want me to update your "/home/user/.google_authenticator" file? (y/n) y | |
| 69 | ``` | |
| 70 | ||
| 71 | ```txt | |
| 72 | OUTPUT | |
| 73 | ||
| 74 | Do you want to disallow multiple uses of the same authentication | |
| 75 | token? This restricts you to one login about every 30s, but it increases | |
| 76 | your chances to notice or even prevent man-in-the-middle attacks (y/n) y | |
| 77 | ``` | |
| 78 | ||
| 79 | ```txt | |
| 80 | OUTPUT | |
| 81 | ||
| 82 | By default, a new token is generated every 30 seconds by the mobile app. | |
| 83 | In order to compensate for possible time-skew between the client and the server, | |
| 84 | we allow an extra token before and after the current time. This allows for a | |
| 85 | time skew of up to 30 seconds between authentication server and client. If you | |
| 86 | experience problems with poor time synchronization, you can increase the window | |
| 87 | from its default size of 3 permitted codes (one previous code, the current | |
| 88 | code, the next code) to 17 permitted codes (the 8 previous codes, the current | |
| 89 | code, and the 8 next codes). This will permit for a time skew of up to 4 minutes | |
| 90 | between client and server. | |
| 91 | Do you want to do so? (y/n) n | |
| 92 | ``` | |
| 93 | ||
| 94 | ```txt | |
| 95 | OUTPUT | |
| 96 | ||
| 97 | If the computer that you are logging into isn't hardened against brute-force | |
| 98 | login attempts, you can enable rate-limiting for the authentication module. | |
| 99 | By default, this limits attackers to no more than 3 login attempts every 30s. | |
| 100 | Do you want to enable rate-limiting? (y/n) y | |
| 101 | ``` | |
| 102 | ||
| 103 | ### Non-Interactive Method | |
| 104 | ||
| 105 | If you need to do this quickly, know your responses to the prompts, or are | |
| 106 | setting this up for numerous users, the non-interactive method can be much | |
| 107 | faster: | |
| 108 | ||
| 109 | ```sh | |
| 110 | google-authenticator -t -d -f -r 3 -R 30 -w 3 | |
| 111 | ``` | |
| 112 | ||
| 113 | The options referenced above are as follows: | |
| 114 | ||
| 115 | ```txt | |
| 116 | google-authenticator [<options>] | |
| 117 | -h, --help Print this message | |
| 118 | -c, --counter-based Set up counter-based (HOTP) verification | |
| 119 | -t, --time-based Set up time-based (TOTP) verification | |
| 120 | -d, --disallow-reuse Disallow reuse of previously used TOTP tokens | |
| 121 | -D, --allow-reuse Allow reuse of previously used TOTP tokens | |
| 122 | -f, --force Write file without first confirming with user | |
| 123 | -l, --label=<label> Override the default label in "otpauth://" URL | |
| 124 | -i, --issuer=<issuer> Override the default issuer in "otpauth://" URL | |
| 125 | -q, --quiet Quiet mode | |
| 126 | -Q, --qr-mode={NONE,ANSI,UTF8} QRCode output mode | |
| 127 | -r, --rate-limit=N Limit logins to N per every M seconds | |
| 128 | -R, --rate-time=M Limit logins to N per every M seconds | |
| 129 | -u, --no-rate-limit Disable rate-limiting | |
| 130 | -s, --secret=<file> Specify a non-standard file location | |
| 131 | -S, --step-size=S Set interval between token refreshes | |
| 132 | -w, --window-size=W Set window of concurrently valid codes | |
| 133 | -W, --minimal-window Disable window of concurrently valid codes | |
| 134 | -e, --emergency-codes=N Number of emergency codes to generate | |
| 135 | ``` | |
| 136 | ||
| 137 | This fully configures the authenticator, saves it to a file, and then outputs | |
| 138 | the secret key, QR code, and recovery codes. (If you add the flag `-q`, then | |
| 139 | there won’t be any output). If you use this command in an automated fashion, | |
| 140 | make sure your script captures the secret key and/or recovery codes and makes | |
| 141 | them available to the user. | |
| 142 | ||
| 143 | ## PAM Configuration Settings | |
| 144 | ||
| 145 | Once you've enabled the T/OTP and have it saved to an MFA app on your phone or | |
| 146 | other device, open the PAM `sshd` file: | |
| 147 | ||
| 148 | ```sh | |
| 149 | sudo nano /etc/pam.d/sshd | |
| 150 | ``` | |
| 151 | ||
| 152 | You need to do two things in this file. First, add the following lines to the | |
| 153 | bottom of the file: | |
| 154 | ||
| 155 | ```config | |
| 156 | auth required pam_google_authenticator.so nullok | |
| 157 | auth required pam_permit.so | |
| 158 | ``` | |
| 159 | ||
| 160 | Second, comment-out the following line near the top of the file. | |
| 161 | ||
| 162 | If you leave this line uncommented, every SSH login attempt will ask for the | |
| 163 | following three authentication factors: | |
| 164 | ||
| 165 | 1. Publickey | |
| 166 | 2. Password | |
| 167 | 3. T/OTP code | |
| 168 | ||
| 169 | ```config | |
| 170 | #@include common-auth | |
| 171 | ``` | |
| 172 | ||
| 173 | ## SSH Configuration Settings | |
| 174 | ||
| 175 | Finally, edit the `sshd_config` file again: | |
| 176 | ||
| 177 | ```sh | |
| 178 | sudo nano /etc/ssh/sshd_config | |
| 179 | ``` | |
| 180 | ||
| 181 | You'll need to change `ChallengeResponseAuthentication` to yes and add the | |
| 182 | `AuthenticationMethods` line to the bottom of the file. | |
| 183 | ||
| 184 | ```config | |
| 185 | ChallengeResponseAuthentication yes | |
| 186 | AuthenticationMethods publickey,password publickey,keyboard-interactive | |
| 187 | ``` | |
| 188 | ||
| 189 | Finally, restart the `ssh` service: | |
| 190 | ||
| 191 | ```sh | |
| 192 | sudo systemctl restart sshd.service | |
| 193 | ``` | |
| 194 | ||
| 195 | The next time you log in, you should be greeted with a verification code | |
| 196 | request! | |
blog/2022-04-02-nginx-reverse-proxy.org added +226
| @@ -0,0 +1,226 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-04-02 | |
| 3 | title = "Set-Up a Reverse Proxy with Nginx" | |
| 4 | description = "Learn how to set-up a reverse proxy server with Nginx on Ubuntu." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## What is a Reverse Proxy? | |
| 9 | ||
| 10 | A reverse proxy is a server that is placed between local servers or services and | |
| 11 | clients/users (e.g., the internet). The reverse proxy intercepts all requests | |
| 12 | from clients at the network edge and uses its configuration files to determine | |
| 13 | where each request should be sent. | |
| 14 | ||
| 15 | ### A Brief Example | |
| 16 | ||
| 17 | For example, let's say that I run three servers in my home: | |
| 18 | ||
| 19 | - Server_01 (`example.com`) | |
| 20 | - Server_02 (`service01.example.com`) | |
| 21 | - Server_03 (`service02.example.com`) | |
| 22 | ||
| 23 | I also run a reverse proxy in my home that intercepts all public traffic: | |
| 24 | ||
| 25 | - Reverse Proxy | |
| 26 | ||
| 27 | Assume that I have a domain name (`example.com`) that allows clients to request | |
| 28 | websites or services from my home servers. | |
| 29 | ||
| 30 | In this case, the reverse proxy will intercept all traffic from `example.com` | |
| 31 | that enters my network and determine if the client is requesting valid data, | |
| 32 | based on my configuration. | |
| 33 | ||
| 34 | If the user is requesting `example.com` and my configuration files say that | |
| 35 | Server_01 holds that data, Nginx will send the user to Server_01. If I were to | |
| 36 | change the configuration so that `example.com` is routed to Server_02, that same | |
| 37 | user would be sent to Server_02 instead. | |
| 38 | ||
| 39 | ```txt | |
| 40 | ┌──────┐ ┌───────────┐ | |
| 41 | │ User │─┐ ┌──► Server_01 │ | |
| 42 | └──────┘ │ │ └───────────┘ | |
| 43 | │ ┌──────────┐ ┌───────────────┐ │ ┌───────────┐ | |
| 44 | ├────► Internet ├───► Reverse Proxy ├─────├──► Server_02 │ | |
| 45 | │ └──────────┘ └───────────────┘ │ └───────────┘ | |
| 46 | ┌──────┐ │ │ ┌───────────┐ | |
| 47 | │ User │─┘ └──► Server_03 │ | |
| 48 | └──────┘ └───────────┘ | |
| 49 | ``` | |
| 50 | ||
| 51 | ## Reverse Proxy Options | |
| 52 | ||
| 53 | There are a lot of options when it comes to reverse proxy servers, so I'm just | |
| 54 | going to list a few of the options I've heard recommended over the last few | |
| 55 | years: | |
| 56 | ||
| 57 | - [Nginx](https://nginx.com) | |
| 58 | - [Caddy](https://caddyserver.com) | |
| 59 | - [Traefik](https://traefik.io/) | |
| 60 | - [HAProxy](https://www.haproxy.org/) | |
| 61 | - [Squid](https://ubuntu.com/server/docs/proxy-servers-squid) | |
| 62 | ||
| 63 | In this post, we will be using Nginx as our reverse proxy, running on Ubuntu | |
| 64 | Server 20.04.4 LTS. | |
| 65 | ||
| 66 | ## Nginx Reverse Proxy Example | |
| 67 | ||
| 68 | ### Local Applications | |
| 69 | ||
| 70 | You may be like me and have a lot of applications running on your local network | |
| 71 | that you'd like to expose publicly with a domain. | |
| 72 | ||
| 73 | In my case, I have services running in multiple Docker containers within a | |
| 74 | single server and want a way to visit those services from anywhere with a URL. | |
| 75 | For example, on my local network, [Dashy](https://dashy.to) runs through port | |
| 76 | 4000 (`localhost:4000`) and | |
| 77 | [Uptime Kuma](https://github.com/louislam/uptime-kuma) runs through port 3001 | |
| 78 | (`localhost:3001`). | |
| 79 | ||
| 80 | In order to expose these services to the public, I will need to do the | |
| 81 | following: | |
| 82 | ||
| 83 | 1. Set up DNS records for a domain or subdomain (one per service) to point | |
| 84 | toward the IP address of the server. | |
| 85 | 2. Open up the server network's HTTP and HTTPS ports (80 & 443) so that the | |
| 86 | reverse proxy can accept traffic and determine where to send it. | |
| 87 | 3. Install the reverse proxy software. | |
| 88 | 4. Configure the reverse proxy to recognize which service should get traffic | |
| 89 | from any of the domains or subdomains. | |
| 90 | ||
| 91 | ### Step 1: DNS Configuration | |
| 92 | ||
| 93 | To start, update your DNS configuration so that you have an `A` record for each | |
| 94 | domain or subdomain. | |
| 95 | ||
| 96 | The `A` records should point toward the public IP address of the server. If you | |
| 97 | don't know the public IP address, log in to the server and run the following | |
| 98 | command: | |
| 99 | ||
| 100 | ```sh | |
| 101 | curl ifconfig.co | |
| 102 | ``` | |
| 103 | ||
| 104 | In the DNS example below, `xxx.xxx.xxx.xxx` is the public IP address of the | |
| 105 | server. | |
| 106 | ||
| 107 | ```config | |
| 108 | example.com A xxx.xxx.xxx.xxx | |
| 109 | uptime.example.com A xxx.xxx.xxx.xxx | |
| 110 | dashy.example.com A xxx.xxx.xxx.xxx | |
| 111 | www CNAME example.com | |
| 112 | ``` | |
| 113 | ||
| 114 | Finally, ensure the DNS has propagated correctly with | |
| 115 | [DNS Checker](https://dnschecker.org) by entering your domains or subdomains in | |
| 116 | the search box and ensuring the results are showing the correct IP address. | |
| 117 | ||
| 118 | ### Step 2: Open Network Ports | |
| 119 | ||
| 120 | This step will be different depending on which router you have in your home. If | |
| 121 | you're not sure, try to visit [192.168.1.1](http://192.168.1.1) in your browser. | |
| 122 | Login credentials are usually written on a sticker somewhere on your | |
| 123 | modem/router. | |
| 124 | ||
| 125 | Once you're able to log in to your router, find the Port Forwarding settings. | |
| 126 | You will need to forward ports `80` and `443` to whichever machine is | |
| 127 | running the reverse proxy. | |
| 128 | ||
| 129 | In my case, the table below shows the port-forwarding rules I've created. | |
| 130 | In this table, `xxx.xxx.xxx.xxx` is the local device IP of the reverse proxy | |
| 131 | server, it will probably be an IP between `192.168.1.1` and `192.168.1.255`. | |
| 132 | ||
| 133 | | NAME | FROM | PORT | DEST PORT/IP | ENABLED | | |
| 134 | |:-----:|:----:|:----:|:---------------:|:-------:| | |
| 135 | | HTTP | \* | 80 | xxx.xxx.xxx.xxx | TRUE | | |
| 136 | | HTTPS | \* | 443 | xxx.xxx.xxx.xxx | TRUE | | |
| 137 | ||
| 138 | Once configured, these rules will direct all web traffic to your reverse proxy. | |
| 139 | ||
| 140 | ### Step 3: Nginx Installation | |
| 141 | ||
| 142 | To install Nginx, simply run the following command: | |
| 143 | ||
| 144 | ```sh | |
| 145 | sudo apt install nginx | |
| 146 | ``` | |
| 147 | ||
| 148 | If you have a firewall enabled, open up ports `80` and `443` on your server so | |
| 149 | that Nginx can accept web traffic from the router. | |
| 150 | ||
| 151 | For example, if you want to use `ufw` for web traffic and SSH, run the following | |
| 152 | commands: | |
| 153 | ||
| 154 | ```sh | |
| 155 | sudo ufw allow 'Nginx Full' | |
| 156 | sudo ufw allow SSH | |
| 157 | sudo ufw enable | |
| 158 | ``` | |
| 159 | ||
| 160 | ### Step 4: Nginx Configuration | |
| 161 | ||
| 162 | Now that we have domains pointing toward the server, the only step left is to | |
| 163 | configure the reverse proxy to direct traffic from domains to local services. | |
| 164 | ||
| 165 | To start, you'll need to create a configuration file for each domain in | |
| 166 | `/etc/nginx/sites-available/`. They will look identical except for the | |
| 167 | `server_name` variable and the `proxy_pass` port. | |
| 168 | ||
| 169 | Dashy: | |
| 170 | ||
| 171 | ```sh | |
| 172 | nano /etc/nginx/sites-available/dashy.example.com | |
| 173 | ``` | |
| 174 | ||
| 175 | ```config | |
| 176 | server { | |
| 177 | listen 80; | |
| 178 | server_name dashy.example.com; | |
| 179 | ||
| 180 | location / { | |
| 181 | proxy_pass http://localhost:4000; | |
| 182 | } | |
| 183 | } | |
| 184 | ``` | |
| 185 | ||
| 186 | Uptime: | |
| 187 | ||
| 188 | ```sh | |
| 189 | nano /etc/nginx/sites-available/uptime.example.com | |
| 190 | ``` | |
| 191 | ||
| 192 | ```config | |
| 193 | server { | |
| 194 | listen 80; | |
| 195 | server_name uptime.example.com; | |
| 196 | ||
| 197 | location / { | |
| 198 | proxy_pass http://localhost:3001; | |
| 199 | } | |
| 200 | } | |
| 201 | ``` | |
| 202 | ||
| 203 | Once the configuration files are created, you will need to enable them with the | |
| 204 | `symlink` command: | |
| 205 | ||
| 206 | ```sh | |
| 207 | sudo ln -s /etc/nginx/sites-available/dashy.example.com /etc/nginx/sites-enabled/ | |
| 208 | ``` | |
| 209 | ||
| 210 | Voilà! Your local services should now be available through their URLs. | |
| 211 | ||
| 212 | ## HTTPS with Certbot | |
| 213 | ||
| 214 | If you've followed along, you'll notice that your services are only available | |
| 215 | via HTTP (not HTTPS). | |
| 216 | ||
| 217 | If you want to enable HTTPS for your new domains, you will need to generate | |
| 218 | SSL/TLS certificates for them. The easiest way to generate certificates on Nginx | |
| 219 | is [Certbot](https://certbot.eff.org): | |
| 220 | ||
| 221 | ```sh | |
| 222 | sudo apt install snapd; sudo snap install core; sudo snap refresh core | |
| 223 | sudo snap install --classic certbot | |
| 224 | sudo ln -s /snap/bin/certbot /usr/bin/certbot | |
| 225 | sudo certbot --nginx | |
| 226 | ``` | |
blog/2022-04-09-pinetime.org added +148
| @@ -0,0 +1,148 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-04-09 | |
| 3 | title = "PineTime: An Open-Source SmartWatch" | |
| 4 | description = "Exploring the PineTime, an open-source smartwatch created by PINE64." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## PineTime Product Information | |
| 9 | ||
| 10 | ### Overview | |
| 11 | ||
| 12 | The [PineTime](https://www.pine64.org/pinetime/) is an open-source smartwatch, | |
| 13 | created by [PINE64](https://www.pine64.org). Originally announced in September | |
| 14 | 2019, this ARM-based watch is a fantastic option for users who want the benefits | |
| 15 | of a modern smartwatch with the backing of open-source components and software. | |
| 16 | ||
| 17 | ### Product Specifications | |
| 18 | ||
| 19 | I won't dive into too many details that you can find on | |
| 20 | [the product page](https://www.pine64.org/pinetime/), but I wanted to point out | |
| 21 | the prices for each watch and the primary functions: | |
| 22 | ||
| 23 | #### Price: | |
| 24 | ||
| 25 | - $26.99 (Sealed) | |
| 26 | - $24.99 (Dev Kit) | |
| 27 | - $51.98 (One Sealed + One Dev Kit) | |
| 28 | ||
| 29 | #### Primary Functionality: | |
| 30 | ||
| 31 | - Clock (+ Smartphone Sync) | |
| 32 | - Pedometer | |
| 33 | - Heart Rate Monitor | |
| 34 | - Sleep Monitor | |
| 35 | - Calories burned | |
| 36 | - Messaging | |
| 37 | - Smartphone Notifications | |
| 38 | - Media Controls | |
| 39 | ||
| 40 | ## Unboxing | |
| 41 | ||
| 42 | Now, my PineTime was ordered on 2022-02-17, shipped on 2022-02-22, and was | |
| 43 | delivered on 2022-03-23. | |
| 44 | With the current delays on shipping times around the world (and the | |
| 45 | semiconductor shortage), a month for delivery from China seems reasonable to me. | |
| 46 | ||
| 47 | The packaging is simple, and the watch comes with instructions, technical | |
| 48 | information, the watch, and a charger (it does not include a USB wall adapter). | |
| 49 | ||
| 50 | The watch itself was able to turn on immediately when I pulled it out of the | |
| 51 | box, but the battery was depleted and required charging right away. | |
| 52 | ||
| 53 |  | |
| 54 | ||
| 55 | ## Software | |
| 56 | ||
| 57 | ### Watch OS: InfiniTime | |
| 58 | ||
| 59 | While turning on the watch for the first time, some of the main design choices | |
| 60 | you can see in the watch OS, | |
| 61 | [InfiniTime](https://wiki.pine64.org/wiki/InfiniTime), are: | |
| 62 | ||
| 63 | - A square bezel, not too thin against the sides of the watch. | |
| 64 | - A simple, rubber band. | |
| 65 | - Basic font and screen pixel design. | |
| 66 | - Swipe gestures to access other screens. | |
| 67 | ||
| 68 |  | |
| 69 | ||
| 70 | The OS itself is fantastic in terms of functionality for me. It does exactly | |
| 71 | what a smartwatch should do - track time, steps, heart rates, and connect to | |
| 72 | another smart device, without being overly burdensome to the user. | |
| 73 | ||
| 74 | My only gripe so far is that it's _really_ difficult to swipe to different | |
| 75 | screens, such as pulling down the notification tray. I'm not sure if this is an | |
| 76 | OS or hardware issue, but it makes it quite hard to quickly move around the | |
| 77 | screens. | |
| 78 | ||
| 79 | However, my absolute favorite design choice is that the button the side turns | |
| 80 | the screen on and off and tilting/waving my wrist doesn't accidentally turn on | |
| 81 | the screen. With other watches, I absolutely hated not being able to turn off | |
| 82 | the raise-to-wake or wave features (i.e., blinding myself while wearing a watch | |
| 83 | at night because I moved my arm). | |
| 84 | ||
| 85 | ### iOS App: InfiniLink | |
| 86 | ||
| 87 | Since I am using iOS as my primary mobile device OS, I am using the | |
| 88 | [InfiniLink](https://github.com/xan-m/InfiniLink) app to connect my watch. | |
| 89 | ||
| 90 | This app provides the following for PineTime owners: | |
| 91 | ||
| 92 | - Firmware updates | |
| 93 | - Steps | |
| 94 | - Charts | |
| 95 | - Notifications | |
| 96 | ||
| 97 | I mashed up a few screenshots to show off the home page, menu, firmware update, | |
| 98 | and notification test screens: | |
| 99 | ||
| 100 |  | |
| 101 | ||
| 102 | Another big feature of InfiniLink is the ability to track pedometer steps in a | |
| 103 | collection of beautiful graphs, with the option to change your step goal and | |
| 104 | add in manual steps. | |
| 105 | ||
| 106 |  | |
| 107 | ||
| 108 | Finally, there are charts to display the battery percentage and heart rates over | |
| 109 | time. This area also comes with an option to clear data. | |
| 110 | ||
| 111 |  | |
| 112 | ||
| 113 | ## Final Thoughts | |
| 114 | ||
| 115 | ### Pros | |
| 116 | ||
| 117 | After wearing my watch for a few weeks, I have mostly positive thoughts about | |
| 118 | the watch so far. | |
| 119 | In the past, I have owned smartwatches by FitBit, Fossil, Apple, etc. - | |
| 120 | **but I prefer the PineTime over all of those watches**. | |
| 121 | ||
| 122 | The PineTime strips out all the unnecessary features and performs the | |
| 123 | functions that it provides effectively and efficiently. | |
| 124 | ||
| 125 | The battery life is amazing on this device. By default, the watch seems to last | |
| 126 | anywhere from a few days to a week before dying. | |
| 127 | ||
| 128 | And of course, it's open source and backed by some of the most dedicated | |
| 129 | enthusiasts and developers I've seen. Watching the Matrix channel, forums, and | |
| 130 | website have been exciting to see. | |
| 131 | ||
| 132 | ### Cons | |
| 133 | ||
| 134 | If I had to complain about anything, it would simply be the small bugs in some | |
| 135 | features that can be contributed to the companion apps more than the watch | |
| 136 | itself. | |
| 137 | ||
| 138 | A regular user would want native notification support out-of-the-box, which is | |
| 139 | the biggest item not working for me at the moment. | |
| 140 | ||
| 141 | My only other complaint is that the battery indicator on the watch doesn't seem | |
| 142 | accurate when it's nearing depletion - it seems that there's a bit of battery | |
| 143 | life left and then my watch is dead very suddenly after. This could just be me | |
| 144 | misinterpreting the battery level icons, but it has fooled me a few times into | |
| 145 | thinking I had more battery left than I actually did. | |
| 146 | ||
| 147 | Other than those small items, I really do love this watch and am glad I replaced | |
| 148 | my Apple Watch with the PineTime. | |
blog/2022-06-01-ditching-cloudflare.org added +89
| @@ -0,0 +1,89 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-06-01 | |
| 3 | title = "Ditching Cloudflare for Njalla" | |
| 4 | description = "After spending a few years jumping around to different DNS hosts, I'm finally ditching Cloudflare for good." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Registrar | |
| 9 | ||
| 10 | After spending a year or so using Cloudflare for DNS only - no proxying or | |
| 11 | applications - I spent the last few months using Cloudflare Tunnels and | |
| 12 | Cloudflare Access to protect my self-hosted websites and applications via their | |
| 13 | proxy traffic model. | |
| 14 | ||
| 15 | However, I have never liked using Cloudflare due to their increasingly large | |
| 16 | share of control over web traffic, as well as their business model of being a | |
| 17 | MITM for all of your traffic. | |
| 18 | ||
| 19 | So, as of today, I have switched over to [Njalla](https://njal.la) as my | |
| 20 | registrar and DNS manager. I was able to easily transfer my domains over | |
| 21 | rapidly, with only one domain taking more than 15-30 minutes to propagate. | |
| 22 | ||
| 23 | ~~I do still have two domains sitting at Cloudflare for the moment while I | |
| 24 | decide if they're worth the higher rates (one domain is 30€ and the other is | |
| 25 | 45€).~~ | |
| 26 | ||
| 27 | > **Update (2022.06.03)**: I ended up transferring my final two domains over to | |
| 28 | > Njalla, clearing my Cloudflare account of personal data, and deleting the | |
| 29 | > Cloudflare account entirely. _I actually feel relieved to have moved on to a | |
| 30 | > provider I trust._ | |
| 31 | ||
| 32 | ## DNS | |
| 33 | ||
| 34 | As noted above, I'm using Njalla exclusively for DNS configurations on my | |
| 35 | domains. | |
| 36 | ||
| 37 | However, the transfer process was not ideal. As soon as the domains transferred | |
| 38 | over, I switched the nameservers from Cloudflare to Njalla and lost most of the | |
| 39 | associated DNS records. So, the majority of the time spent during the migration | |
| 40 | was simply re-typing all the DNS records back in one-by-one. | |
| 41 | ||
| 42 | This would be much simpler if I were able to edit the plain-text format of the | |
| 43 | DNS configuration. I was able to do that at a past registrar (perhaps it was | |
| 44 | [Gandi.net](https://gandi.net/)?) and it made life a lot easier. | |
| 45 | ||
| 46 | ### Dynamic DNS Updates | |
| 47 | ||
| 48 | I have built an easy Python script to run (or set-up in `cron` to run | |
| 49 | automatically) that will check my server's IPv4 and IPv6, compare it to Njalla, | |
| 50 | and update the DNS records if they don't match. You can see the full script and | |
| 51 | process in my other post: | |
| 52 | [Updating Dynamic DNS with Njalla API](/blog/njalla-dns-api/). | |
| 53 | ||
| 54 | I haven't used this other method, but I do know that you can create `Dynamic` | |
| 55 | DNS records with Njalla that | |
| 56 | [work for updating dynamic subdomains](https://njal.la/docs/ddns/). | |
| 57 | ||
| 58 | ### Njalla's DNS Tool | |
| 59 | ||
| 60 | One neat upside to Njalla is that they have a | |
| 61 | [DNS lookup tool](https://check.njal.la/dns/) that provides a lot of great | |
| 62 | information for those of you (AKA: me) who hate using the `dig` command. | |
| 63 | ||
| 64 | This was very useful for monitoring a couple of my transferred domains to see | |
| 65 | when the changes in nameservers, records, and DNSSEC went into effect. | |
| 66 | ||
| 67 | ## Tunnel | |
| 68 | ||
| 69 | Cloudflare Tunnel is a service that acts as a reverse-proxy (hosted on | |
| 70 | Cloudflare's servers) and allowed me to mask the private IP address of the | |
| 71 | server hosting my various websites and apps. | |
| 72 | ||
| 73 | However, as I was moving away from Cloudflare, I was not able to find a suitable | |
| 74 | replacement that was both inexpensive and simple. So, I simply went back to | |
| 75 | hosting [my own reverse proxy with Nginx](/blog/set-up-nginx-reverse-proxy/). | |
| 76 | With the recent additions of Unifi hardware in my server/network rack, I am much | |
| 77 | more protected against spam and malicious attacks at the network edge than I was | |
| 78 | before I switched to Cloudflare. | |
| 79 | ||
| 80 | ## Access | |
| 81 | ||
| 82 | Cloudflare Access, another app I used in combination with Cloudflare Tunnel, | |
| 83 | provided an authentication screen that required you to enter valid credentials | |
| 84 | before Cloudflare would forward you to the actual website or app (if the | |
| 85 | website/app has their own authentication, you'd then have to authenticate a | |
| 86 | second time). | |
| 87 | ||
| 88 | I did not replace this service with anything since I only host a handful of | |
| 89 | non-sensitive apps that don't require duplicate authentication. | |
blog/2022-06-04-njalla-dns-api.org added +205
| @@ -0,0 +1,205 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-06-04 | |
| 3 | title = "Dynamic DNS with Njalla API" | |
| 4 | description = "Learn how to update Njalla DNS records automatically with a simple Python script." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Njalla's API | |
| 9 | ||
| 10 | As noted in my recent post about | |
| 11 | [switching to Njalla from Cloudflare](/blog/ditching-cloudflare/), I was | |
| 12 | searching for a way to replace my very easy-to-use bash script to | |
| 13 | [update Cloudflare's DNS via their API](/blog/cloudflare-dns-api/). | |
| 14 | ||
| 15 | To reiterate what I said in those posts, this is a common necessity for those of | |
| 16 | us who have non-static IP addresses that can change at any moment due to ISP | |
| 17 | policy. | |
| 18 | ||
| 19 | In order to keep a home server running smoothly, the server admin needs to have | |
| 20 | a process to constantly monitor their public IP address and update their | |
| 21 | domain's DNS records if it changes. | |
| 22 | ||
| 23 | This post explains how to use Python to update Njalla's DNS records whenever a | |
| 24 | machine's public IP address changes. | |
| 25 | ||
| 26 | ### Creating a Token | |
| 27 | ||
| 28 | To use Njalla's API, you will first need to create a token that will be used to | |
| 29 | authenticate you every time you call the API. Luckily, this is very easy to do | |
| 30 | if you have an account with Njalla. | |
| 31 | ||
| 32 | Simply go the [API Settings](https://njal.la/settings/api/) page and click the | |
| 33 | `Add Token` button. Next, enter a name for the token and click `Add`. | |
| 34 | ||
| 35 | Finally, click the `Manage` button next to your newly created token and copy the | |
| 36 | `API Token` field. | |
| 37 | ||
| 38 | ### Finding the Correct API Request | |
| 39 | ||
| 40 | Once you have a token, you're ready to call the Njalla API for any number of | |
| 41 | requests. For a full listing of available requests, see the | |
| 42 | [Njalla API Documentation](https://njal.la/api/). | |
| 43 | ||
| 44 | For this demo, we are using the `list-records` and `edit-record` requests. | |
| 45 | ||
| 46 | The `list-records` request requires the following payload to be sent when | |
| 47 | calling the API: | |
| 48 | ||
| 49 | ```txt | |
| 50 | params: { | |
| 51 | domain: string | |
| 52 | } | |
| 53 | ``` | |
| 54 | ||
| 55 | The `edit-record` request requires the following payload to be sent when calling | |
| 56 | the API: | |
| 57 | ||
| 58 | ```txt | |
| 59 | params: { | |
| 60 | domain: string | |
| 61 | id: int | |
| 62 | content: string | |
| 63 | } | |
| 64 | ``` | |
| 65 | ||
| 66 | ## Server Set-Up | |
| 67 | ||
| 68 | To create this script, we will be using Python. By default, I use Python 3 on my | |
| 69 | servers, so please note that I did not test this in Python 2, and I do not know | |
| 70 | if Python 2 will work for this. | |
| 71 | ||
| 72 | ### Creating the Script | |
| 73 | ||
| 74 | First, find a suitable place to create your script. Personally, I just create a | |
| 75 | directory called `ddns` in my home directory: | |
| 76 | ||
| 77 | ```sh | |
| 78 | mkdir ~/ddns | |
| 79 | ``` | |
| 80 | ||
| 81 | Next, create a Python script file: | |
| 82 | ||
| 83 | ```sh | |
| 84 | nano ~/ddns/ddns.py | |
| 85 | ``` | |
| 86 | ||
| 87 | The following code snippet is quite long, so I won't go into depth on each part. | |
| 88 | However, I suggest you read through the entire script before running it; it is | |
| 89 | quite simple and contains comments to help explain each code block. | |
| 90 | ||
| 91 | :warning: **Note**: You will need to update the following variables for this to | |
| 92 | work: | |
| 93 | ||
| 94 | - `token`: This is the Njalla API token you created earlier. | |
| 95 | - `user_domain`: This is the top-level domain you want to modify. | |
| 96 | - `include_subdomains`: Set this to `True` if you also want to modify | |
| 97 | subdomains found under the TLD. | |
| 98 | - `subdomains`: If `include_subdomains` = `True`, you can include your list of | |
| 99 | subdomains to be modified here. | |
| 100 | ||
| 101 | ```python | |
| 102 | #!/usr/bin/python | |
| 103 | # -*- coding: utf-8 -*- | |
| 104 | # Import Python modules | |
| 105 | ||
| 106 | from requests import get | |
| 107 | import requests | |
| 108 | import json | |
| 109 | ||
| 110 | # Set global variables | |
| 111 | ||
| 112 | url = 'https://njal.la/api/1/' | |
| 113 | token = '<your-api-token>' | |
| 114 | user_domain = 'example.com' | |
| 115 | include_subdomains = True | |
| 116 | subdomains = ['one', 'two'] | |
| 117 | ||
| 118 | ||
| 119 | # Main API call function | |
| 120 | ||
| 121 | def njalla(method, **params): | |
| 122 | headers = {'Authorization': 'Njalla ' + token} | |
| 123 | response = requests.post(url, json={'method': method, | |
| 124 | 'params': params}, headers=headers).json() | |
| 125 | if 'result' not in response: | |
| 126 | raise Exception('API Error', response) | |
| 127 | return response['result'] | |
| 128 | ||
| 129 | ||
| 130 | # Gather all DNS records for a domain | |
| 131 | ||
| 132 | def get_records(domain): | |
| 133 | return njalla('list-records', domain=user_domain) | |
| 134 | ||
| 135 | ||
| 136 | # Update a DNS record for a domain | |
| 137 | ||
| 138 | def update_record(domain, record_id, record_content): | |
| 139 | return njalla('edit-record', domain=domain, id=record_id, | |
| 140 | content=record_content) | |
| 141 | ||
| 142 | ||
| 143 | # Get public IP addresses | |
| 144 | ||
| 145 | ipv4 = get('https://api.ipify.org').text | |
| 146 | print('IPv4: {}'.format(ipv4)) | |
| 147 | ipv6 = get('https://api64.ipify.org').text | |
| 148 | print('IPv6: {}'.format(ipv6)) | |
| 149 | ||
| 150 | # Call API to get all DNS records | |
| 151 | ||
| 152 | data = get_records(user_domain) | |
| 153 | ||
| 154 | # Loop through records and check if each one is IPv4 (A) or IPv6 (AAAA) | |
| 155 | # Update only if DNS is different from server IP | |
| 156 | ||
| 157 | for record in data['records']: | |
| 158 | if record['name'] == '@' or (include_subdomains and record['name'] \ | |
| 159 | in subdomains): | |
| 160 | if record['type'] == 'A': | |
| 161 | if record['content'] == ipv4: | |
| 162 | print(record['type'], 'record for', record['name'], | |
| 163 | 'already matches public IPv4 address. Skipping...' | |
| 164 | ) | |
| 165 | else: | |
| 166 | print('IPv4 of', ipv4, | |
| 167 | 'does not match Njalla\'s value of', | |
| 168 | record['content'], '. Updating...') | |
| 169 | update_record(user_domain, record['id'], ipv4) | |
| 170 | elif record['type'] == 'AAAA': | |
| 171 | if record['content'] == ipv6: | |
| 172 | print(record['type'], 'record for', record['name'], | |
| 173 | 'already matches public IPv6 address. Skipping...' | |
| 174 | ) | |
| 175 | else: | |
| 176 | print('IPv6 of', ipv6, | |
| 177 | 'does not match Njalla\'s value of', | |
| 178 | record['content'], '. Updating...') | |
| 179 | update_record(user_domain, record['id'], ipv6) | |
| 180 | ``` | |
| 181 | ||
| 182 | ### Running the Script | |
| 183 | ||
| 184 | Once you've created the script and are ready to test it, run the following | |
| 185 | command: | |
| 186 | ||
| 187 | ```sh | |
| 188 | python3 ~/ddns/ddns.py | |
| 189 | ``` | |
| 190 | ||
| 191 | ### Setting the Script to Run Automatically | |
| 192 | ||
| 193 | To make sure the scripts run automatically, add it to the `cron` file so that it | |
| 194 | will run on a schedule. To do this, open the `cron` file: | |
| 195 | ||
| 196 | ```sh | |
| 197 | crontab -e | |
| 198 | ``` | |
| 199 | ||
| 200 | In the cron file, paste the following at the bottom of the editor in order to | |
| 201 | check the IP every five minutes: | |
| 202 | ||
| 203 | ```sh | |
| 204 | */5 * * * * python3 /home/<your_username>/ddns/ddns.py | |
| 205 | ``` | |
blog/2022-06-07-freshrss.org added +240
| @@ -0,0 +1,240 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-06-07 | |
| 3 | title = "Self-Hosting FreshRSS" | |
| 4 | description = "A detailed description of my process to self-host FreshRSS on Ubuntu." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Why RSS? | |
| 9 | ||
| 10 | After noticing that I have collected 50+ blogs as bookmarks, I decided to | |
| 11 | migrate back to using RSS feeds to stay up-to-date with my favorite websites. | |
| 12 | Using RSS allows me to read all of these posts in a single app (on both mobile & | |
| 13 | desktop) and allows me to be notified when new posts are available. | |
| 14 | ||
| 15 | However, I ran into one issue: syncing subscriptions and read/unread posts | |
| 16 | across devices. Since I want to be able to easily read on both mobile and | |
| 17 | desktop, I decided to look for a self-hosted RSS solution. | |
| 18 | ||
| 19 | Thus, I found [FreshRSS](https://www.freshrss.org/) and was able to successfully | |
| 20 | install it on my server in about 30 minutes. | |
| 21 | ||
| 22 | ## Documentation | |
| 23 | ||
| 24 | While it's certainly not robust, the | |
| 25 | [FreshRSS documentation](https://freshrss.github.io/FreshRSS/) is helpful for | |
| 26 | figuring out basic information about the service. | |
| 27 | ||
| 28 | However, I wanted to install this service as a Docker container and stumbled | |
| 29 | across the | |
| 30 | [Docker README](https://github.com/FreshRSS/FreshRSS/tree/edge/Docker) within | |
| 31 | the GitHub repository. | |
| 32 | ||
| 33 | This README was the documentation I actually needed. However, as you'll see | |
| 34 | below, I still had to manually edit one file (`config.php`) to access the API | |
| 35 | externally via my RSS apps. | |
| 36 | ||
| 37 | ## Installation | |
| 38 | ||
| 39 | ### DNS | |
| 40 | ||
| 41 | The first step, as required by any external web service, was assigning a domain | |
| 42 | name to use. I chose to use a subdomain, like `rss.example.com`. | |
| 43 | ||
| 44 | To assign this, I created an `A` record in my DNS settings with the IPv4 address | |
| 45 | of the server and an `AAAA` record with the IPv6 address of the server. Note: | |
| 46 | assigning an IPv6 (`AAAA`) record is optional, but I like to enable IPV6 for my | |
| 47 | services. | |
| 48 | ||
| 49 | ```config | |
| 50 | rss.example.com A xxx.xxx.xxx.xxx | |
| 51 | rss.example.com AAAA xxxx:xxxx: ... :xxxx | |
| 52 | ``` | |
| 53 | ||
| 54 | ### Docker | |
| 55 | ||
| 56 | I initially tried to set up a `docker-compose.yml` file with a `.env` file | |
| 57 | because I prefer to have a file I can look back at later to see how I initially | |
| 58 | started the container, but it simply wouldn't work for me. I'm not sure why, but | |
| 59 | I assume I wasn't telling `docker-compose` where the `.env` file was. | |
| 60 | ||
| 61 | Regardless, I chose to simply run the service with `docker run`. See the | |
| 62 | following command for my `docker run` configuration: | |
| 63 | ||
| 64 | ```sh | |
| 65 | sudo docker run -d --restart unless-stopped --log-opt max-size=10m \ | |
| 66 | -p 8080:80 \ | |
| 67 | -e TZ=America/Chicago \ | |
| 68 | -e 'CRON_MIN=1,31' \ | |
| 69 | -v freshrss_data:/var/www/FreshRSS/data \ | |
| 70 | -v freshrss_extensions:/var/www/FreshRSS/extensions \ | |
| 71 | --name freshrss \ | |
| 72 | freshrss/freshrss | |
| 73 | ``` | |
| 74 | ||
| 75 | This started the container successfully and allowed me to visit the FreshRSS | |
| 76 | instance at `localhost:8080`. | |
| 77 | ||
| 78 | ### Fresh RSS Set-Up | |
| 79 | ||
| 80 | I **HIGHLY** suggest that you set up your user account prior to | |
| 81 | exposing this service to the public. It's unlikely that someone is trying to | |
| 82 | access the exact domain or IP/port you're assigning here, but as soon as you | |
| 83 | expose this service, the first person to open the URL will be able to create the | |
| 84 | admin user. | |
| 85 | ||
| 86 | In order to set up your FreshRSS service, open the `localhost:8080` URL in your | |
| 87 | browser (you may need to use a local IP instead of `localhost` if you're | |
| 88 | accessing the page from a different machine on the network - e.g., | |
| 89 | `192.168.1.20:8080`). | |
| 90 | ||
| 91 | Once the page loads, set up your default user with a strong username and | |
| 92 | password. You may also choose to configure other settings prior to exposing this | |
| 93 | service. | |
| 94 | ||
| 95 | ### Nginx Reverse-Proxy | |
| 96 | ||
| 97 | In order to access this service outside my home, I needed to set up a | |
| 98 | reverse-proxy to connect `localhost:8080` to `rss.example.com`. | |
| 99 | ||
| 100 | First, I created a new Nginx configuration file: | |
| 101 | ||
| 102 | ```sh | |
| 103 | sudo nano /etc/nginx/sites-available/rss.example.com | |
| 104 | ``` | |
| 105 | ||
| 106 | Within the config file, I pasted the following code: | |
| 107 | ||
| 108 | ```config | |
| 109 | upstream freshrss { | |
| 110 | server 127.0.0.1:8080; | |
| 111 | keepalive 64; | |
| 112 | } | |
| 113 | ||
| 114 | server { | |
| 115 | server_name rss.example.com; | |
| 116 | listen 80; | |
| 117 | ||
| 118 | location / { | |
| 119 | # The final `/` is important. | |
| 120 | proxy_pass http://localhost:8080/; | |
| 121 | add_header X-Frame-Options SAMEORIGIN; | |
| 122 | add_header X-XSS-Protection "1; mode=block"; | |
| 123 | proxy_redirect off; | |
| 124 | proxy_buffering off; | |
| 125 | proxy_set_header Host $host; | |
| 126 | proxy_set_header X-Real-IP $remote_addr; | |
| 127 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
| 128 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 129 | proxy_set_header X-Forwarded-Port $server_port; | |
| 130 | proxy_read_timeout 90; | |
| 131 | ||
| 132 | # Forward the Authorization header for the Google Reader API. | |
| 133 | proxy_set_header Authorization $http_authorization; | |
| 134 | proxy_pass_header Authorization; | |
| 135 | } | |
| 136 | } | |
| 137 | ``` | |
| 138 | ||
| 139 | Finally, restart Nginx and you will be able to access your service via HTTP: | |
| 140 | ||
| 141 | ```sh | |
| 142 | sudo systemctl restart nginx.service | |
| 143 | ``` | |
| 144 | ||
| 145 | ### HTTPS | |
| 146 | ||
| 147 | However, I don't want to access my RSS feeds via HTTP. I want it available only | |
| 148 | via HTTPS. In order to do this, I ran the [certbot](https://certbot.eff.org/) | |
| 149 | program to generate SSL certificates for me: | |
| 150 | ||
| 151 | ```sh | |
| 152 | sudo certbot --nginx | |
| 153 | ``` | |
| 154 | ||
| 155 | This process will automatically generate an SSL certificate for you and modify | |
| 156 | the Nginx configuration file to include a redirect from HTTP to HTTPS. | |
| 157 | ||
| 158 | ## Post-Installation Fixes | |
| 159 | ||
| 160 | At this point, we have a functional FreshRSS website, available from anywhere | |
| 161 | and secured with HTTPS. However, attempting to connect this service to an RSS | |
| 162 | app resulted in many errors regarding unavailable URLs and incorrect | |
| 163 | credentials. | |
| 164 | ||
| 165 | ### API Set-Up | |
| 166 | ||
| 167 | First, you need to open your user profile in FreshRSS (`Settings` > `Profile`) | |
| 168 | and set an API password in the field at the bottom. This is the password you | |
| 169 | will need to provide to your RSS apps. | |
| 170 | ||
| 171 | Once that is set and saved, click the link below the API password field to open | |
| 172 | the API check tool. It should look something like `https://localhost:8080/api/` | |
| 173 | or `https://rss.example.com/api/`. | |
| 174 | ||
| 175 | Within this page, you _should_ see your correct external URL and "PASS" at the | |
| 176 | bottom of each API type. | |
| 177 | This would mean everything is set up correctly, and you can now move on and | |
| 178 | login to any RSS apps that support self-hosted options. | |
| 179 | ||
| 180 | In my case, the URL showed an internal URL and I had a warning that the | |
| 181 | `base_url` variable may be misconfigured. If this is the case, see the next | |
| 182 | section for a fix. | |
| 183 | ||
| 184 | ### Base URL Fix | |
| 185 | ||
| 186 | In order to fix the `base_url` for the API, I opened up my docker container with | |
| 187 | the following command: | |
| 188 | ||
| 189 | ```sh | |
| 190 | sudo docker exec -it freshrss bash | |
| 191 | ``` | |
| 192 | ||
| 193 | Within this container, update the packages and install an editor: | |
| 194 | ||
| 195 | ```sh | |
| 196 | apt-get update | |
| 197 | apt-get install nano | |
| 198 | ``` | |
| 199 | ||
| 200 | Finally, open up `config.php` in the `data` directory: | |
| 201 | ||
| 202 | ```sh | |
| 203 | nano data/config.php | |
| 204 | ``` | |
| 205 | ||
| 206 | Within `config.php`, you will need to update the `base_url` variable and update | |
| 207 | it to match your external URL. In my case, I simply commented-out the incorrect | |
| 208 | URL with `//` and added the correct one on a new line: | |
| 209 | ||
| 210 | ```php | |
| 211 | <?php | |
| 212 | return array ( | |
| 213 | ... | |
| 214 | // 'base_url' => 'http://localhost:8080', | |
| 215 | 'base_url' => 'https://rss.example.com', | |
| 216 | ... | |
| 217 | ) | |
| 218 | > | |
| 219 | ``` | |
| 220 | ||
| 221 | You can now exit the file with `Ctrl + x`, press `y` to save the file, and then | |
| 222 | click `Enter` to keep the same file name. | |
| 223 | ||
| 224 | Finally, just exit out of the docker container: | |
| 225 | ||
| 226 | ```sh | |
| 227 | exit | |
| 228 | ``` | |
| 229 | ||
| 230 | Next, just restart the container: | |
| 231 | ||
| 232 | ```sh | |
| 233 | sudo docker restart freshrss | |
| 234 | ``` | |
| 235 | ||
| 236 | Voilà! Your API check should now "PASS" and you should be able to use one of the | |
| 237 | API URLs in your RSS apps. | |
| 238 | ||
| 239 | In my case, I use [NetNewsWire](https://netnewswire.com) on my desktop and | |
| 240 | phone. | |
blog/2022-06-16-terminal-lifestyle.org added +202
| @@ -0,0 +1,202 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-06-16 | |
| 3 | title = "A Terminal Lifestyle" | |
| 4 | description = "A detailed description on how I'm living my digital life in the terminal." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Text-Based Simplicity | |
| 8 | ||
| 9 | I've detailed my views on web-based minimalism and related topics in other | |
| 10 | posts throughout the years; e.g., JavaScript/CSS bloat slowing down websites | |
| 11 | that are essentially a text document. However, I have never really expanded | |
| 12 | beyond talking about the web and describing how I focus on minimizing | |
| 13 | distractions in other digital environments. | |
| 14 | ||
| 15 | This post is going to set the baseline for how I *try* to live my digital life. | |
| 16 | It does not necessarily get into my physical life, which is often harder to | |
| 17 | control and contain all the noise in our modern world. | |
| 18 | ||
| 19 | While there are new things to do every day in our digital world, I find that | |
| 20 | keeping a core set of values and interests can ground you and keep you mindful | |
| 21 | of *why* you are participating in the digital world. For example, if - at your | |
| 22 | core - you have no interest in what strangers think about random topics, it | |
| 23 | would be unwise to start participating in social media. However, I am someone | |
| 24 | who has been dragged in by effective advertising to participate in communities | |
| 25 | that I realize I do not care for. | |
| 26 | ||
| 27 | I won't dive much further into explaining the philosophy of all this, but I will | |
| 28 | link a few helpful articles that may pique your interest if you're in search of | |
| 29 | more meaningful experiences: | |
| 30 | ||
| 31 | - [Mindfulness](https://en.wikipedia.org/wiki/Mindfulness) | |
| 32 | - [Minimalism](https://en.wikipedia.org/wiki/Minimalism) | |
| 33 | - [Stoicism](https://en.wikipedia.org/wiki/Stoicism) | |
| 34 | ||
| 35 | ## Living Life in the Terminal | |
| 36 | ||
| 37 | My personal approach to reducing digital distractions and increasing my focus on | |
| 38 | the task at hand is to use a terminal for as much as I possibly can. | |
| 39 | ||
| 40 | Most days, this means that I have a few tabs open constantly in my terminal: | |
| 41 | ||
| 42 | 1. A web browser | |
| 43 | 2. A chat client | |
| 44 | 3. An email client | |
| 45 | 4. An RSS feed reader | |
| 46 | 5. A local shell for navigating my computer's files | |
| 47 | 6. A remote shell for managing servers and other machines | |
| 48 | ||
| 49 | Beyond this, I rarely open other tabs or GUI applications, unless absolutely | |
| 50 | necessary. If you look, you may be surprised what can be accomplished in the | |
| 51 | terminal. | |
| 52 | ||
| 53 | For example, I have moved my music and entertainment downloads to the | |
| 54 | terminal, along with my device VPN connections. I am exploring options for | |
| 55 | moving my RSS subscriptions to something like [Newsboat](https://newsboat.org/), | |
| 56 | so that I can read my daily articles without all the fuss. | |
| 57 | ||
| 58 | Now that we have some examples out of the way, let's dive into the | |
| 59 | specifics. | |
| 60 | ||
| 61 | ### Browsing the Web | |
| 62 | ||
| 63 | I'm going to start off with a hard topic for those who prefer to live in the | |
| 64 | terminal: web browsing. This task is made hard mostly by websites and web apps | |
| 65 | that require JavaScript to run. The other difficult part is that if you're using | |
| 66 | a text-based browser, that means images won't load (hopefully that's obvious). | |
| 67 | ||
| 68 | I am using [Lynx](https://lynx.invisible-island.net), a text-based browser that | |
| 69 | runs quickly and easily in the terminal. Lynx allows me to browser most websites | |
| 70 | by simply typing `g` and then typing in the URL I want. | |
| 71 | ||
| 72 |  | |
| 73 | ||
| 74 | If you need a search engine while in Lynx, I recommend [DuckDuckGo | |
| 75 | (Lite)](https://lite.duckduckgo.com/lite/), which allows you to search the web | |
| 76 | using their text-only interface. | |
| 77 | ||
| 78 |  | |
| 80 | ||
| 81 | Eventually, you will run into websites that don't work (or are just too ugly | |
| 82 | and messy) in a text-only mode, and you'll be forced to switch over to a GUI | |
| 83 | browser to look at that site. Personally, I don't mind this as it doesn't happen | |
| 84 | as often as I thought it would. | |
| 85 | ||
| 86 | The only time I need to do this is when I want to browse an image/video-focused | |
| 87 | webpage or if I need to log in to a site, and it doesn't support a text-only | |
| 88 | login page. | |
| 89 | For example, I am able to easily log in to [Sourcehut](https://sr.ht) in lynx. | |
| 90 | ||
| 91 | ### Chatting with Friends | |
| 92 | ||
| 93 | After web browsing activities, my main form of terminal communication is Matrix. | |
| 94 | I use the [gomuks](https://docs.mau.fi/gomuks/) client currently. | |
| 95 | ||
| 96 | This was incredibly easy to install on macOS (but I will need to see if it'll | |
| 97 | be just as easy on Linux when my new laptop arrives): | |
| 98 | ||
| 99 | ```sh | |
| 100 | brew install gomuks | |
| 101 | ``` | |
| 102 | ||
| 103 | Once you launch gomuks, it will sync and require your username and password to | |
| 104 | login. After doing so, the only problem I ran into was verifying my gomuks | |
| 105 | client so that I could participate in rooms with E2EE. | |
| 106 | ||
| 107 | Finally, I was able to verify the session by opening the Element desktop app (I | |
| 108 | assume you can do this in the browser and mobile app too, but I'm not sure) and | |
| 109 | manually verifying myself with this process: | |
| 110 | ||
| 111 | 1. Open the Element desktop app | |
| 112 | 2. Open a room I was a member of | |
| 113 | 3. Open the `Room Info` pane | |
| 114 | 4. Open the `People` menu and search for myself | |
| 115 | 5. Click on my profile name | |
| 116 | 6. Click on the session link under the `Security` section and follow the | |
| 117 | prompts to manually verify the session | |
| 118 | ||
| 119 | Overall, I like gomuks and am able to enjoy all the features I was using in | |
| 120 | Element. The only hiccup I have occurred is manually downloading images to view | |
| 121 | them, which can be annoying. | |
| 122 | ||
| 123 |  | |
| 124 | ||
| 125 | ||
| 126 | ||
| 127 | Moving email to the terminal has been the hardest of the tasks for me. Unlike | |
| 128 | web browsing, where I can simply decide to not look at a website that does not | |
| 129 | work in the terminal, I cannot simply ignore emails sent to me. | |
| 130 | ||
| 131 | Personally, I am experimenting with [neomutt](https://neomutt.org/) as a | |
| 132 | potential email client. | |
| 133 | ||
| 134 | However, this requires a **TON** of configuration and tweaking to get right. | |
| 135 | Even when I was able to set up neomutt, configure my email account, and | |
| 136 | customize a few personal preferences, a lot of emails still do not display | |
| 137 | correctly (mostly due to HTML and images). | |
| 138 | ||
| 139 | I won't get into the details of configuring `neomutt`; I mostly followed this | |
| 140 | blog post: [Email in the Terminal: Configuring | |
| 141 | Neomutt](https://gideonwolfe.com/posts/workflow/neomutt/intro/). | |
| 142 | ||
| 143 | Finally, I have yet to figure out how to connect my GPG keys to `neomutt`, but | |
| 144 | that's a problem for another day. | |
| 145 | ||
| 146 | ### RSS Feed Reader | |
| 147 | ||
| 148 | I have just started using [Newsboat](https://newsboat.org/) to read articles in | |
| 149 | my terminal and have found quick success with it. | |
| 150 | ||
| 151 | I'll show you a quick screenshot first: | |
| 152 | ||
| 153 |  | |
| 154 | ||
| 155 | The configuration was super easy for this app; I simply installed the app, | |
| 156 | created a file for URLs, and imported my OPML subscriptions that I had exported | |
| 157 | out of my old feed reader: | |
| 158 | ||
| 159 | ```sh | |
| 160 | brew install newsboat | |
| 161 | ``` | |
| 162 | ||
| 163 | ```sh | |
| 164 | touch ~/.newsboat/urls | |
| 165 | ``` | |
| 166 | ||
| 167 | ```sh | |
| 168 | newsboat -i=my_subscriptions.opml | |
| 169 | ``` | |
| 170 | ||
| 171 | ### Writing & Programming | |
| 172 | ||
| 173 | Unfortunately, the weak link in my terminal-based environment right now is my | |
| 174 | grasp of the possibilities of editing files within a shell. | |
| 175 | ||
| 176 | I am used to the easy extensions found in VSCodium and Kate, so I am slowly | |
| 177 | learning how to mold the default editing tools to my needs. Currently, this | |
| 178 | means I am using `nano` with the following configuration: | |
| 179 | ||
| 180 | ```config | |
| 181 | set breaklonglines | |
| 182 | set autoindent | |
| 183 | set linenumbers | |
| 184 | set tabstospaces | |
| 185 | set tabsize 2 | |
| 186 | set fill 80 | |
| 187 | ``` | |
| 188 | ||
| 189 | This configuration allows nano to automatically hard-wrap lines at 80 | |
| 190 | characters, autoindent the wrapped lines (if the previous line was indented), | |
| 191 | use 2 spaces per tab, and display line numbers within each file I open. | |
| 192 | ||
| 193 | I am currently looking to see if `vim` or `emacs` would be more useful for my | |
| 194 | current needs, but I'm not in any rush, so I don't expect to find an answer | |
| 195 | anytime soon. | |
| 196 | ||
| 197 | With my current life demands, I am not programming at the moment and have not | |
| 198 | explored the best terminal set-up for programming. However, I have seen many | |
| 199 | peers find success configuring `vim` and `emacs`, so that's where I will start | |
| 200 | when I pick my projects back up. | |
| 201 | ||
| 202 |  | |
blog/2022-06-22-daily-poetry.org added +213
| @@ -0,0 +1,213 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-06-22 | |
| 3 | title = "Daily Plaintext Poetry via Email" | |
| 4 | description = "A quick tutorial showing how to use Python and the PoetryDB API to email daily poetry." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Source Code | |
| 9 | ||
| 10 | I don't want to bury the lede here, so if you'd like to see the full source | |
| 11 | code I use to email myself plaintext poems daily, visit the repository: | |
| 12 | [daily-poem](https://git.sr.ht/~cmc/daily-poem/). | |
| 13 | ||
| 14 | ## My Daily Dose of Poetry | |
| 15 | ||
| 16 | Most of my programming projects are small, random projects that are made | |
| 17 | strictly to fix some small problem I have or enhance my quality of life. | |
| 18 | ||
| 19 | In this case, I was looking for a simply and easy way to get a daily dose of | |
| 20 | literature or poetry to read in the mornings. | |
| 21 | ||
| 22 | However, I don't want to sign up for a random mailing list on just any website. | |
| 23 | I also don't want to have to work to find the reading content each morning, as I | |
| 24 | know I would simply give up and stop reading daily. | |
| 25 | ||
| 26 | Thus, I found a way to deliver poetry to myself in plain-text format, on a daily | |
| 27 | basis, and scheduled to deliver automatically. | |
| 28 | ||
| 29 | ## Prerequisites | |
| 30 | ||
| 31 | This solution uses Python and email, so the following process requires the | |
| 32 | following to be installed: | |
| 33 | ||
| 34 | 1. An SMTP server, which can be as easy as installing `mailutils` if you're on | |
| 35 | a Debian-based distro. | |
| 36 | 2. Python (& pip!) | |
| 37 | 3. The following Python packages: `email`, `smtplib`, `json`, and `requests` | |
| 38 | ||
| 39 | ## Breaking Down the Logic | |
| 40 | ||
| 41 | I want to break down the logic for this program, as it's quite simple and | |
| 42 | informational. | |
| 43 | ||
| 44 | ### Required Packages | |
| 45 | ||
| 46 | This program starts with a simple import of the required packages, so I wanted | |
| 47 | to explain why each package is used: | |
| 48 | ||
| 49 | ```py | |
| 50 | from email.mime.text import MIMEText # Required for translating MIMEText | |
| 51 | import smtplib # Required to process the SMTP mail delivery | |
| 52 | import json # Required to parse the poetry API results | |
| 53 | import requests # Required to send out a request to the API | |
| 54 | ``` | |
| 55 | ||
| 56 | ### Sending the API Request | |
| 57 | ||
| 58 | Next, we need to actually send the API request. In my case, I'm calling a random | |
| 59 | poem from the entire API. If you want, you can call specific poems or authors | |
| 60 | from this API. | |
| 61 | ||
| 62 | ```py | |
| 63 | json_data = requests.get('https://poetrydb.org/random').json() | |
| 64 | ``` | |
| 65 | ||
| 66 | This gives us the following result in JSON: | |
| 67 | ||
| 68 | ```json | |
| 69 | [ | |
| 70 | { | |
| 71 | "title": "Sonnet XXII: With Fools and Children", | |
| 72 | "author": "Michael Drayton", | |
| 73 | "lines": [ | |
| 74 | "To Folly", | |
| 75 | "", | |
| 76 | "With fools and children, good discretion bears;", | |
| 77 | "Then, honest people, bear with Love and me,", | |
| 78 | "Nor older yet, nor wiser made by years,", | |
| 79 | "Amongst the rest of fools and children be;", | |
| 80 | "Love, still a baby, plays with gauds and toys,", | |
| 81 | "And, like a wanton, sports with every feather,", | |
| 82 | "And idiots still are running after boys,", | |
| 83 | "Then fools and children fitt'st to go together.", | |
| 84 | "He still as young as when he first was born,", | |
| 85 | "No wiser I than when as young as he;", | |
| 86 | "You that behold us, laugh us not to scorn;", | |
| 87 | "Give Nature thanks you are not such as we.", | |
| 88 | "Yet fools and children sometimes tell in play", | |
| 89 | "Some, wise in show, more fools indeed than they." | |
| 90 | ], | |
| 91 | "linecount": "15" | |
| 92 | } | |
| 93 | ] | |
| 94 | ``` | |
| 95 | ||
| 96 | ### Parsing the API Results | |
| 97 | ||
| 98 | In order to parse this into a readable format, we need to use the `json` package | |
| 99 | and extract the fields we want. In the example below, I am grabbing every field | |
| 100 | presented by the API. | |
| 101 | ||
| 102 | For the actual poem content, we need to loop over each line in the `lines` | |
| 103 | variable since each line is a separate string by default. | |
| 104 | ||
| 105 | > You *could* also extract the title or author and make another call out to the | |
| 106 | > API to avoid having to build the plaintext poem with a loop, but it just | |
| 107 | > doesn't make sense to me to send multiple requests when we can create a simple | |
| 108 | > loop on our local machine to work with the data we already have. | |
| 109 | > | |
| 110 | > For | |
| 111 | > [example](https://poetrydb.org/title/Sonnet%20XXII:%20With%20Fools%20and%20Children/lines.text), | |
| 112 | > look at the raw data response of this link to see the poem's lines returned | |
| 113 | > in plaintext. | |
| 114 | ||
| 115 | ```py | |
| 116 | title = json_data[0]['title'] | |
| 117 | author = json_data[0]['author'] | |
| 118 | line_count = json_data[0]['linecount'] | |
| 119 | lines = '' | |
| 120 | for line in json_data[0]['lines']: | |
| 121 | lines = lines + line + "\n" | |
| 122 | ``` | |
| 123 | ||
| 124 | ### Composing the Email | |
| 125 | ||
| 126 | Now that I have all the data I need, I just need to compose it into a message | |
| 127 | and prepare the message metadata. | |
| 128 | ||
| 129 | For my daily email, I want to see the title of the poem first, followed by the | |
| 130 | author, then a blank line, and finally the full poem. This code snippet combines | |
| 131 | that data and packages it into a MIMEText container, ready to be emailed. | |
| 132 | ||
| 133 | ```py | |
| 134 | msg_body = title + "\n" + author + "\n\n" + lines | |
| 135 | msg = MIMEText(msg_body) | |
| 136 | ``` | |
| 137 | ||
| 138 | Before we send the email, we need to prepare the metadata (subject, from, to, | |
| 139 | etc.): | |
| 140 | ||
| 141 | ```py | |
| 142 | sender_email = 'example@server.local' | |
| 143 | recipient_emails = ['user@example.com'] | |
| 144 | msg['Subject'] = 'Your Daily Poem (' + line_count + ' lines)' | |
| 145 | msg['From'] = sender_email | |
| 146 | msg['To'] = recipient_email | |
| 147 | ``` | |
| 148 | ||
| 149 | ### Sending the Email | |
| 150 | ||
| 151 | Now that I have everything ready to be emailed, the last step is to simply | |
| 152 | connect to an SMTP server and send the email out to the recipients. In my case, | |
| 153 | I installed `mailutils` on Ubuntu and let my SMTP server be `localhost`. | |
| 154 | ||
| 155 | ```py | |
| 156 | smtp_server = 'localhost' | |
| 157 | s = smtplib.SMTP(smtp_server) | |
| 158 | s.sendmail(sender_email, recipient_emails, msg.as_string()) | |
| 159 | s.quit() | |
| 160 | ``` | |
| 161 | ||
| 162 | ## The Result! | |
| 163 | ||
| 164 | Instead of including a screenshot, I've copied the contents of the email that | |
| 165 | was delivered to my inbox below since I set this process up in plaintext format. | |
| 166 | ||
| 167 | ```txt | |
| 168 | Date: Wed, 22 Jun 2022 14:37:19 +0000 (UTC) | |
| 169 | From: REDACTED | |
| 170 | To: REDACTED | |
| 171 | Subject: Your Daily Poem (36 lines) | |
| 172 | MIME-Version: 1.0 | |
| 173 | Content-Transfer-Encoding: 8bit | |
| 174 | Content-Type: text/plain; charset=utf-8 | |
| 175 | ||
| 176 | Sonnet XXII: With Fools and Children | |
| 177 | Michael Drayton | |
| 178 | ||
| 179 | With fools and children, good discretion bears; | |
| 180 | Then, honest people, bear with Love and me, | |
| 181 | Nor older yet, nor wiser made by years, | |
| 182 | Amongst the rest of fools and children be; | |
| 183 | Love, still a baby, plays with gauds and toys, | |
| 184 | And, like a wanton, sports with every feather, | |
| 185 | And idiots still are running after boys, | |
| 186 | Then fools and children fitt'st to go together. | |
| 187 | He still as young as when he first was born, | |
| 188 | No wiser I than when as young as he; | |
| 189 | You that behold us, laugh us not to scorn; | |
| 190 | Give Nature thanks you are not such as we. | |
| 191 | Yet fools and children sometimes tell in play | |
| 192 | Some, wise in show, more fools indeed than they. | |
| 193 | ``` | |
| 194 | ||
| 195 | ## Scheduling the Daily Email | |
| 196 | ||
| 197 | Last, but not least, is scheduling this Python script with `crontab`. To | |
| 198 | schedule a script to run daily, you can add it to the `crontab` file. To do | |
| 199 | this, open `crontab` in editing mode: | |
| 200 | ||
| 201 | ```sh | |
| 202 | crontab -e | |
| 203 | ``` | |
| 204 | ||
| 205 | In the file, simply paste the following snippet at the bottom of the file and | |
| 206 | ensure that the file path is correctly pointing to wherever you saved your | |
| 207 | Python script: | |
| 208 | ||
| 209 | ```config | |
| 210 | 0 8 * * * python3 /home/<your_user>/dailypoem/main.py | |
| 211 | ``` | |
| 212 | ||
| 213 | We have now set up the script and scheduled it to run daily at 08:00! | |
blog/2022-06-24-fedora-i3.org added +153
| @@ -0,0 +1,153 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-06-24 | |
| 3 | title = "Rebooting My Love Affair with Linux" | |
| 4 | description = "A quick recap on my migration from macOS to Fedora i3." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## Leaving macOS | |
| 9 | ||
| 10 | As I noted [in a recent post](/blog/foss-macos-apps), I have been planning on | |
| 11 | migrating from macOS back to a Linux-based OS. I am happy to say that I have | |
| 12 | finally completed my migration and am now stuck in the wonderful world of Linux | |
| 13 | again. | |
| 14 | ||
| 15 | My decision to leave macOS really came down to just a few important things: | |
| 16 | ||
| 17 | - Apple Security (Gatekeeper) restricting me from running any software I want. | |
| 18 | Even if you disable Gatekeeper and allow software to bypass the rest of the | |
| 19 | device installation security, you still have to repeat that process every time | |
| 20 | the allowed software is updated. | |
| 21 | - macOS sends out nearly constant connections, pings, telemetry, etc. to a | |
| 22 | myriad of mysterious Apple services. I'm not even going to dive into how many | |
| 23 | macOS apps have constant telemetry on, as well. | |
| 24 | - Lastly, I just *really* missed the customization and freedom that comes with | |
| 25 | Linux. Being able to switch to entirely new kernel, OS, or desktop within | |
| 26 | minutes is a freedom I took for granted when I switched to macOS. | |
| 27 | ||
| 28 | Now that I've covered macOS, I'm going to move on to more exciting topics: my | |
| 29 | personal choice of OS, DE, and various customizations I'm using. | |
| 30 | ||
| 31 | ## Fedora | |
| 32 | ||
| 33 | After trying a ton of distros (I think I booted and tested around 20-25 | |
| 34 | distros), I finally landed on [Fedora Linux](https://getfedora.org/). I have | |
| 35 | quite a bit of experience with Fedora and enjoy the `dnf` package manager. | |
| 36 | Fedora allows me to keep up-to-date with recent software (I'm looking at you, | |
| 37 | Debian), but still provides a level of stability you don't find in every distro. | |
| 38 | ||
| 39 | In a very close second place was Arch Linux, as well as its spin-off: Garuda | |
| 40 | Linux (Garuda w/ sway is *beautiful*). Arch is great for compatibility and the | |
| 41 | massive community it has, but I have just never had the time to properly sit | |
| 42 | down and learn the methodology behind their packaging systems. | |
| 43 | ||
| 44 | Basically, everything else I tested was unacceptable in at least one way or | |
| 45 | another. | |
| 46 | Void (`glibc`) was great, but doesn't support all the software I need. | |
| 47 | Slackware worked well as a tui, but I wasn't skilled enough to get a tiling | |
| 48 | window manager (WM) working on it. | |
| 49 | ||
| 50 | ### i3 | |
| 51 | ||
| 52 | One of the reasons I settled on Fedora is that it comes with an official i3 | |
| 53 | spin. Being able to use a tiling WM, such as i3 or sway, is one of the biggest | |
| 54 | things I wanted to do as soon as I adopted Linux again. | |
| 55 | ||
| 56 | I will probably set up a dotfile repository soon, so that I don't lose any | |
| 57 | of my configurations, but nothing big has been configured thus far. | |
| 58 | ||
| 59 | The two main things I have updated in i3wm are natural scrolling and binding my | |
| 60 | brightness keys to the `brightnessctl` program. | |
| 61 | ||
| 62 | #### Natural Scrolling | |
| 63 | ||
| 64 | You can enable natural scrolling by opening the following file: | |
| 65 | ||
| 66 | ```sh | |
| 67 | sudo nano /usr/share/X11/xorg.conf.d/40-libinput.conf | |
| 68 | ``` | |
| 69 | ||
| 70 | Within the `40-libinput.conf` file, find the following input sections and enable | |
| 71 | the natural scrolling option. | |
| 72 | ||
| 73 | This is the `pointer` section: | |
| 74 | ||
| 75 | ```conf | |
| 76 | Section "InputClass" | |
| 77 | Identifier "libinput pointer catchall" | |
| 78 | MatchIsPointer "on" | |
| 79 | MatchDevicePath "/dev/input/event*" | |
| 80 | Driver "libinput" | |
| 81 | Option "NaturalScrolling" "True" | |
| 82 | EndSection | |
| 83 | ``` | |
| 84 | ||
| 85 | This is the `touchpad` section: | |
| 86 | ||
| 87 | ```conf | |
| 88 | Section "InputClass" | |
| 89 | Identifier "libinput touchpad catchall" | |
| 90 | MatchIsTouchpad "on" | |
| 91 | MatchDevicePath "/dev/input/event*" | |
| 92 | Driver "libinput" | |
| 93 | Option "NaturalScrolling" "True" | |
| 94 | EndSection | |
| 95 | ``` | |
| 96 | ||
| 97 | #### Enabling Brightness Keys | |
| 98 | ||
| 99 | Likewise, enabling brightness key functionality is as simple as binding the keys | |
| 100 | to the `brightnessctl` program. | |
| 101 | ||
| 102 | To do this, open up your i3 config file. Mine is located here: | |
| 103 | ||
| 104 | ```sh | |
| 105 | nano /home/<my-user>/.config/i3/config | |
| 106 | ``` | |
| 107 | ||
| 108 | ```conf | |
| 109 | # Use brightnessctl to adjust brightness. | |
| 110 | bindsym XF86MonBrightnessDown exec --no-startup-id brightnessctl --min-val=2 -q set 3%- | |
| 111 | bindsym XF86MonBrightnessUp exec --no-startup-id brightnessctl -q set 3%+ | |
| 112 | ``` | |
| 113 | ||
| 114 | #### `polybar` | |
| 115 | ||
| 116 | Instead of using the default `i3status` bar, I have opted to use `polybar` | |
| 117 | instead (as you can also see in the screenshot above). | |
| 118 | ||
| 119 | My config for this menu bar is basically just the default settings with modified | |
| 120 | colors and an added battery block to quickly show me the machine's battery info. | |
| 121 | ||
| 122 | #### `alacritty` | |
| 123 | ||
| 124 | Not much to say on this part yet, as I haven't configured it much, but I | |
| 125 | installed `alacritty` as my default terminal, and I am using `zsh` and the | |
| 126 | shell. | |
| 127 | ||
| 128 | ## Software Choices | |
| 129 | ||
| 130 | Again, I'm not going to say much that I haven't said yet in other blog | |
| 131 | posts, so I'll just do a quick rundown of the apps I installed immediately after | |
| 132 | I set up the environment. | |
| 133 | ||
| 134 | Flatpak Apps: | |
| 135 | ||
| 136 | - Cryptomator | |
| 137 | - pCloud | |
| 138 | - Signal | |
| 139 | ||
| 140 | Fedora Packages: | |
| 141 | ||
| 142 | - gomuks | |
| 143 | - neomutt | |
| 144 | - neofetch | |
| 145 | - Firefox | |
| 146 | - uBlock Origin | |
| 147 | - Bitwarden | |
| 148 | - Stylus | |
| 149 | - Privacy Redirect | |
| 150 | ||
| 151 | Other: | |
| 152 | ||
| 153 | - exiftool | |
blog/2022-07-01-git-server.org added +626
| @@ -0,0 +1,626 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-07-01 | |
| 3 | title = "Self-Hosting a Personal Git Server" | |
| 4 | description = "My retrospective on successfully setting up a personal git server at home." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## My Approach to Self-Hosting Git | |
| 8 | ||
| 9 | I have often tried to self-host my Git repositories, but have always fallen | |
| 10 | short when I tried to find a suitable web interface to show on the front-end. | |
| 11 | ||
| 12 | After a few years, I have finally found a combination of methods that allow me | |
| 13 | to easily self-host my projects, view them on the web, and access them from | |
| 14 | anywhere. | |
| 15 | ||
| 16 | Before I dive into the details, I want to state a high-level summary of my | |
| 17 | self-hosted Git approach: | |
| 18 | ||
| 19 | - This method uses the `ssh://` (read & write) and `git://` (read-only) | |
| 20 | protocols for push and pull access. | |
| 21 | - For the `git://` protocol, I create a `git-daemon-export-ok` file in any | |
| 22 | repository that I want to be cloneable by anyone. | |
| 23 | - The web interface I am using (`cgit`) allows simple HTTP cloning by | |
| 24 | default. | |
| 25 | I do not disable this setting as I want beginners to be able to | |
| 26 | clone one of my repositories even if they don't know the proper method. | |
| 27 | - I am not enabling Smart HTTPS for any repositories. Updates to repositories | |
| 28 | must be pushed via SSH. | |
| 29 | - Beyond the actual repository management, I am using `cgit` for the front-end | |
| 30 | web interface. | |
| 31 | - If you use the `scan-path=<path>` configuration in the `cgitrc` | |
| 32 | configuration file to automatically find repositories, you can't exclude a | |
| 33 | repository from `cgit` if it's stored within the path that `cgit` reads. To | |
| 34 | host private repositories, you'd need to set up another directory that `cgit` | |
| 35 | can't read. | |
| 36 | ||
| 37 | ## Assumptions | |
| 38 | ||
| 39 | For the purposes of this walkthrough, I am assuming you have a URL | |
| 40 | (`git.example.com`) or IP address (`207.84.26.991`) addressed to the server that | |
| 41 | you will be using to host your git repositories. | |
| 42 | ||
| 43 | ## Adding a Git User | |
| 44 | ||
| 45 | In order to use the SSH method associated with git, we will need to add a | |
| 46 | user named `git`. If you have used the SSH method for other git hosting sites, | |
| 47 | you are probably used to the following syntax: | |
| 48 | ||
| 49 | ```sh | |
| 50 | git clone [user@]server:project.git | |
| 51 | ``` | |
| 52 | ||
| 53 | The syntax above is an `scp`-like syntax for using SSH on the `git` user on the | |
| 54 | server to access your repository. | |
| 55 | ||
| 56 | Let's delete any remnants of an old `git` user, if any, and create the new user | |
| 57 | account: | |
| 58 | ||
| 59 | ```sh | |
| 60 | sudo deluser --remove-home git | |
| 61 | sudo adduser git | |
| 62 | ``` | |
| 63 | ||
| 64 | ### Import Your SSH Keys to the Git User | |
| 65 | ||
| 66 | Once the `git` user is created, you will need to copy your public SSH key on | |
| 67 | your local development machine to the `git` user on the server. | |
| 68 | ||
| 69 | If you don't have an SSH key yet, create one with this command: | |
| 70 | ||
| 71 | ```sh | |
| 72 | ssh-keygen | |
| 73 | ``` | |
| 74 | ||
| 75 | Once you create the key pair, the public should be saved to `~/.ssh/id_rsa.pub`. | |
| 76 | ||
| 77 | If your server still has password-based authentication available, you can copy | |
| 78 | it over to your user's home directory like this: | |
| 79 | ||
| 80 | ```sh | |
| 81 | ssh-copy-id git@server | |
| 82 | ``` | |
| 83 | ||
| 84 | Otherwise, copy it over to any user that you can access. | |
| 85 | ||
| 86 | ```sh | |
| 87 | scp ~/.ssh/id_rsa.pub your_user@your_server: | |
| 88 | ``` | |
| 89 | ||
| 90 | Once on the server, you will need to copy the contents into the `git` user's | |
| 91 | `authorized_keys` file: | |
| 92 | ||
| 93 | ```sh | |
| 94 | cat id_rsa.pub > /home/git/.ssh/authorized_keys | |
| 95 | ``` | |
| 96 | ||
| 97 | ### (Optional) Disable Password-Based SSH | |
| 98 | ||
| 99 | If you want to lock down your server and ensure that no one can authenticate in | |
| 100 | via SSH with a password, you will need to edit your SSH configuration. | |
| 101 | ||
| 102 | ```sh | |
| 103 | sudo nano /etc/ssh/sshd_config | |
| 104 | ``` | |
| 105 | ||
| 106 | Within this file, find the following settings and set them to the values I am | |
| 107 | showing below: | |
| 108 | ||
| 109 | ```conf | |
| 110 | PermitRootLogin no | |
| 111 | PasswordAuthentication no | |
| 112 | AuthenticationMethods publickey | |
| 113 | ``` | |
| 114 | ||
| 115 | You may have other Authentication Methods required in your personal set-up, so | |
| 116 | the key here is just to ensure that `AuthenticationMethods` does not allow | |
| 117 | passwords. | |
| 118 | ||
| 119 | ## Setting up the Base Directory | |
| 120 | ||
| 121 | Now that we have set up a `git` user to handle all transport methods, we need | |
| 122 | to set up the directory that we will be using as our base of all repositories. | |
| 123 | ||
| 124 | In my case, I am using `/git` as my source folder. To create this folder and | |
| 125 | assign it to the user we created, execute the following commands: | |
| 126 | ||
| 127 | ```sh | |
| 128 | sudo mkdir /git | |
| 129 | sudo chown -R git:git /git | |
| 130 | ``` | |
| 131 | ||
| 132 | ## Creating a Test Repository | |
| 133 | ||
| 134 | On your server, switch over to the `git` user in order to start managing git | |
| 135 | files. | |
| 136 | ||
| 137 | ```sh | |
| 138 | su git | |
| 139 | ``` | |
| 140 | ||
| 141 | Once logged-in as the `git` user, go to your base directory and create a test | |
| 142 | repository. | |
| 143 | ||
| 144 | ```sh | |
| 145 | cd /git | |
| 146 | mkdir test.git && cd test.git | |
| 147 | git init --bare | |
| 148 | ``` | |
| 149 | ||
| 150 | If you want to make this repo viewable/cloneable to the public via the `git://` | |
| 151 | protocol, you need to create a `git-daemon-export-ok` file inside the | |
| 152 | repository. | |
| 153 | ||
| 154 | ```sh | |
| 155 | touch git-daemon-export-ok | |
| 156 | ``` | |
| 157 | ||
| 158 | ## Change the Login Shell for `git` | |
| 159 | ||
| 160 | To make sure that the `git` user is only used for git operations and nothing | |
| 161 | else, you need to change the user's login shell. To do this, simply use the | |
| 162 | `chsh` command: | |
| 163 | ||
| 164 | ```sh | |
| 165 | sudo chsh git | |
| 166 | ``` | |
| 167 | ||
| 168 | The interactive prompt will ask which shell you want the `git` user to use. You | |
| 169 | must use the following value: | |
| 170 | ||
| 171 | ```sh | |
| 172 | /usr/bin/git-shell | |
| 173 | ``` | |
| 174 | ||
| 175 | Once done, no one will be able to SSH to the `git` user or execute commands | |
| 176 | other than the standard git commands. | |
| 177 | ||
| 178 | ## Opening the Firewall | |
| 179 | ||
| 180 | Don't forget to open up ports on the device firewall and network firewall if | |
| 181 | you want to access these repositories publicly. If you're using default ports, | |
| 182 | forward ports `22` (ssh) and `9418` (git) from your router to your server's IP | |
| 183 | address. | |
| 184 | ||
| 185 | If your server also has a firewall, ensure that the firewall allows the same | |
| 186 | ports that are forwarded from the router. For example, if you use `ufw`: | |
| 187 | ||
| 188 | ```sh | |
| 189 | sudo ufw allow 22 | |
| 190 | sudo ufw allow 9418 | |
| 191 | ``` | |
| 192 | ||
| 193 | ### Non-Standard SSH Ports | |
| 194 | ||
| 195 | If you use a non-standard port for SSH, such as `9876`, you will need to create | |
| 196 | an SSH configuration file on your local development machine in order to connect | |
| 197 | to your server's git repositories. | |
| 198 | ||
| 199 | To do this, you'll need to define your custom port on your client machine in | |
| 200 | your `~/.ssh/config` file: | |
| 201 | ||
| 202 | ```sh | |
| 203 | nano ~/.ssh/config | |
| 204 | ``` | |
| 205 | ||
| 206 | ```conf | |
| 207 | Host git.example.com | |
| 208 | # HostName can be a URL or an IP address | |
| 209 | HostName git.example.com | |
| 210 | Port 9876 | |
| 211 | User git | |
| 212 | ``` | |
| 213 | ||
| 214 | ### Testing SSH | |
| 215 | ||
| 216 | There are two main syntaxes you can use to manage git over SSH: | |
| 217 | ||
| 218 | - `git clone [user@]server:project.git` | |
| 219 | - `git clone ssh://[user@]server/project.git` | |
| 220 | ||
| 221 | I prefer the first, which is an `scp`-like syntax. To test it, try to clone the | |
| 222 | test repository you set up on the server: | |
| 223 | ||
| 224 | ```sh | |
| 225 | git clone git@git.example.com:/git/test.git | |
| 226 | ``` | |
| 227 | ||
| 228 | ## Enabling Read-Only Access | |
| 229 | ||
| 230 | If you want people to be able to clone any repository where you've placed a | |
| 231 | `git-daemon-export-ok` file, you will need to start the git daemon. | |
| 232 | ||
| 233 | To do this on a system with `systemd`, create a service file: | |
| 234 | ||
| 235 | ```sh | |
| 236 | sudo nano /etc/systemd/system/git-daemon.service | |
| 237 | ``` | |
| 238 | ||
| 239 | Inside the `git-daemon.service` file, paste the following: | |
| 240 | ||
| 241 | ```conf | |
| 242 | [Unit] | |
| 243 | Description=Start Git Daemon | |
| 244 | ||
| 245 | [Service] | |
| 246 | ExecStart=/usr/bin/git daemon --reuseaddr --base-path=/git/ /git/ | |
| 247 | ||
| 248 | Restart=always | |
| 249 | RestartSec=500ms | |
| 250 | ||
| 251 | StandardOutput=syslog | |
| 252 | StandardError=syslog | |
| 253 | SyslogIdentifier=git-daemon | |
| 254 | ||
| 255 | User=git | |
| 256 | Group=git | |
| 257 | ||
| 258 | [Install] | |
| 259 | WantedBy=multi-user.target | |
| 260 | ``` | |
| 261 | ||
| 262 | Once created, enable and start the service: | |
| 263 | ||
| 264 | ```sh | |
| 265 | sudo systemctl enable git-daemon.service | |
| 266 | sudo systemctl start git-daemon.service | |
| 267 | ``` | |
| 268 | ||
| 269 | To clone read-only via the `git://` protocol, you can use the following syntax: | |
| 270 | ||
| 271 | ```sh | |
| 272 | git clone git://git.example.com/test.git | |
| 273 | ``` | |
| 274 | ||
| 275 | ## Migrating Repositories | |
| 276 | ||
| 277 | At this point, we have a working git server that works with both SSH and | |
| 278 | read-only access. | |
| 279 | ||
| 280 | For each of the repositories I had hosted a different provider, I executed the | |
| 281 | following commands in order to place a copy on my server as my new source of | |
| 282 | truth: | |
| 283 | ||
| 284 | Server: | |
| 285 | ||
| 286 | ```sh | |
| 287 | su git | |
| 288 | mkdir /git/<REPOSITORY_NAME>.git && cd /git/<REPOSITORY_NAME>.git | |
| 289 | git init --bare | |
| 290 | ||
| 291 | # If you want to make this repo viewable/cloneable to the public | |
| 292 | touch git-daemon-export-ok | |
| 293 | ``` | |
| 294 | ||
| 295 | Client: | |
| 296 | ||
| 297 | ```sh | |
| 298 | git clone git@<PREVIOUS_HOST>:<REPOSITORY_NAME> | |
| 299 | git remote set-url origin git@git.EXAMPLE.COM:/git/<REPOSITORY_NAME>.git | |
| 300 | git push | |
| 301 | ``` | |
| 302 | ||
| 303 | ## Optional Web View: `cgit` | |
| 304 | ||
| 305 | If you want a web viewer for your repositories, you can use various tools, such | |
| 306 | as `gitweb`, `cgit`, or `klaus`. I chose `cgit` due to its simple interface and | |
| 307 | fairly easy set-up (compared to others). Not to mention that the | |
| 308 | [Linux kernel uses `cgit`](https://git.kernel.org/). | |
| 309 | ||
| 310 | ### Docker Compose | |
| 311 | ||
| 312 | Instead of using my previous method of using a `docker run` command, I've | |
| 313 | updated this section to use `docker-compose` instead for an easier installation | |
| 314 | and simpler management and configuration. | |
| 315 | ||
| 316 | In order to use Docker Compose, you will set up a `docker-compose.yml` file to | |
| 317 | automatically connect resources like the repositories, `cgitrc`, and various | |
| 318 | files or folders to the `cgit` container you're creating: | |
| 319 | ||
| 320 | ```sh | |
| 321 | mkdir ~/cgit && cd ~/cgit | |
| 322 | nano docker-compose.yml | |
| 323 | ``` | |
| 324 | ||
| 325 | ```conf | |
| 326 | # docker-compose.yml | |
| 327 | version: '3' | |
| 328 | ||
| 329 | services: | |
| 330 | cgit: | |
| 331 | image: invokr/cgit | |
| 332 | volumes: | |
| 333 | - /git:/git | |
| 334 | - ./cgitrc:/etc/cgitrc | |
| 335 | - ./logo.png:/var/www/htdocs/cgit/logo.png | |
| 336 | - ./favicon.png:/var/www/htdocs/cgit/favicon.png | |
| 337 | - ./filters:/var/www/htdocs/cgit/filters | |
| 338 | ports: | |
| 339 | - "8763:80" | |
| 340 | restart: always | |
| 341 | ``` | |
| 342 | ||
| 343 | Then, just start the container: | |
| 344 | ||
| 345 | ```sh | |
| 346 | sudo docker-compose up -d | |
| 347 | ``` | |
| 348 | ||
| 349 | Once it's finished installing, you can access the site at `<SERVER_IP>:8763` or | |
| 350 | use a reverse-proxy service to forward `cgit` to a URL, such as | |
| 351 | `git.example.com`. See the next section for more details on reverse proxying a | |
| 352 | URL to a local port. | |
| 353 | ||
| 354 | ### Nginx Reverse Proxy | |
| 355 | ||
| 356 | I am using Nginx as my reverse proxy so that the `cgit` Docker container can use | |
| 357 | `git.example.com` as its URL. To do so, I simply created the following | |
| 358 | configuration file: | |
| 359 | ||
| 360 | ```sh | |
| 361 | sudo nano /etc/nginx/sites-available/git.example.com | |
| 362 | ``` | |
| 363 | ||
| 364 | ```conf | |
| 365 | server { | |
| 366 | listen 80; | |
| 367 | server_name git.example.com; | |
| 368 | ||
| 369 | if ($host = git.example.com) { | |
| 370 | return 301 https://$host$request_uri; | |
| 371 | } | |
| 372 | ||
| 373 | return 404; | |
| 374 | } | |
| 375 | ||
| 376 | server { | |
| 377 | server_name git.example.com; | |
| 378 | listen 443 ssl http2; | |
| 379 | ||
| 380 | location / { | |
| 381 | # The final `/` is important. | |
| 382 | proxy_pass http://localhost:8763/; | |
| 383 | add_header X-Frame-Options SAMEORIGIN; | |
| 384 | add_header X-XSS-Protection "1; mode=block"; | |
| 385 | proxy_redirect off; | |
| 386 | proxy_buffering off; | |
| 387 | proxy_set_header Host $host; | |
| 388 | proxy_set_header X-Real-IP $remote_addr; | |
| 389 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
| 390 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 391 | proxy_set_header X-Forwarded-Port $server_port; | |
| 392 | } | |
| 393 | ||
| 394 | # INCLUDE ANY SSL CERTS HERE | |
| 395 | include /etc/letsencrypt/options-ssl-nginx.conf; | |
| 396 | ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; | |
| 397 | } | |
| 398 | ||
| 399 | ``` | |
| 400 | ||
| 401 | Once created, symlink it and restart the web server. | |
| 402 | ||
| 403 | ```sh | |
| 404 | sudo ln -s /etc/nginx/sites-available/git.example.com /etc/nginx/sites-enabled/ | |
| 405 | sudo systemctl restart nginx.service | |
| 406 | ``` | |
| 407 | ||
| 408 | As we can see below, my site at `git.example.com` is available and running: | |
| 409 | ||
| 410 | ### Settings Up Git Details | |
| 411 | ||
| 412 | Once you have `cgit` running, you can add some small details, such as repository | |
| 413 | owners and descriptions by editing the following files within each repository. | |
| 414 | ||
| 415 | Alternatively, you can use the `cgitrc` file to edit these details if you only | |
| 416 | care to edit them for the purpose of seeing them on your website. | |
| 417 | ||
| 418 | The `description` file within the repository on your server will display the | |
| 419 | description online. | |
| 420 | ||
| 421 | ```sh | |
| 422 | cd /git/example.git | |
| 423 | nano description | |
| 424 | ``` | |
| 425 | ||
| 426 | You can add a `[gitweb]` block to the `config` file in order to display the | |
| 427 | owner of the repository. | |
| 428 | ||
| 429 | ```sh | |
| 430 | cd /git/example.git | |
| 431 | nano config | |
| 432 | ``` | |
| 433 | ||
| 434 | ```conf | |
| 435 | [gitweb] | |
| 436 | owner = "YourName" | |
| 437 | ``` | |
| 438 | ||
| 439 | Note that you can ignore the configuration within each repository and simply set | |
| 440 | up this information in the `cgitrc` file, if you want to do it that way. | |
| 441 | ||
| 442 | ### Editing `cgit` | |
| 443 | ||
| 444 | In order to edit certain items within `cgit`, you need to edit the `cgitrc` | |
| 445 | file. | |
| 446 | ||
| 447 | ```sh | |
| 448 | nano ~/cgit/cgitrc | |
| 449 | ``` | |
| 450 | ||
| 451 | Below is an example configuration for `cgitrc`. | |
| 452 | You can find all the configuration options within the [configuration manual] | |
| 453 | (https://git.zx2c4.com/cgit/plain/cgitrc.5.txt). | |
| 454 | ||
| 455 | ```conf | |
| 456 | css=/cgit.css | |
| 457 | logo=/logo.png | |
| 458 | favicon=/favicon.png | |
| 459 | robots=noindex, nofollow | |
| 460 | ||
| 461 | enable-index-links=1 | |
| 462 | enable-commit-graph=1 | |
| 463 | enable-blame=1 | |
| 464 | enable-log-filecount=1 | |
| 465 | enable-log-linecount=1 | |
| 466 | enable-git-config=1 | |
| 467 | ||
| 468 | clone-url=git://git.example.com/$CGIT_REPO_URL ssh://git@git.example.com:/git/$CGIT_REPO_URL | |
| 469 | ||
| 470 | root-title=My Git Website | |
| 471 | root-desc=My personal git repositories. | |
| 472 | ||
| 473 | # Allow download of tar.gz, tar.bz2 and zip-files | |
| 474 | snapshots=tar.gz tar.bz2 zip | |
| 475 | ||
| 476 | ## | |
| 477 | ## List of common mimetypes | |
| 478 | ## | |
| 479 | mimetype.gif=image/gif | |
| 480 | mimetype.html=text/html | |
| 481 | mimetype.jpg=image/jpeg | |
| 482 | mimetype.jpeg=image/jpeg | |
| 483 | mimetype.pdf=application/pdf | |
| 484 | mimetype.png=image/png | |
| 485 | mimetype.svg=image/svg+xml | |
| 486 | ||
| 487 | # Highlight source code | |
| 488 | # source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.sh | |
| 489 | source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py | |
| 490 | ||
| 491 | # Format markdown, restructuredtext, manpages, text files, and html files | |
| 492 | # through the right converters | |
| 493 | about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh | |
| 494 | ||
| 495 | ## | |
| 496 | ## Search for these files in the root of the default branch of repositories | |
| 497 | ## for coming up with the about page: | |
| 498 | ## | |
| 499 | readme=:README.md | |
| 500 | readme=:readme.md | |
| 501 | readme=:README.mkd | |
| 502 | readme=:readme.mkd | |
| 503 | readme=:README.rst | |
| 504 | readme=:readme.rst | |
| 505 | readme=:README.html | |
| 506 | readme=:readme.html | |
| 507 | readme=:README.htm | |
| 508 | readme=:readme.htm | |
| 509 | readme=:README.txt | |
| 510 | readme=:readme.txt | |
| 511 | readme=:README | |
| 512 | readme=:readme | |
| 513 | ||
| 514 | # Repositories | |
| 515 | ||
| 516 | # Uncomment the following line to scan a path instead of adding repositories manually | |
| 517 | # scan-path=/git | |
| 518 | ||
| 519 | ## Test Section | |
| 520 | section=git/test-section | |
| 521 | ||
| 522 | repo.url=test.git | |
| 523 | repo.path=/git/test.git | |
| 524 | repo.readme=:README.md | |
| 525 | repo.owner=John Doe | |
| 526 | repo.desc=An example repository! | |
| 527 | ``` | |
| 528 | ||
| 529 | ### Final Fixes: Syntax Highlighting & README Rendering | |
| 530 | ||
| 531 | After completing my initial install and playing around with it for a few days, I | |
| 532 | noticed two issues: | |
| 533 | ||
| 534 | 1. Syntax highlighting did not work when viewing the source code within a file. | |
| 535 | 2. The `about` tab within a repository was not rendered to HTML. | |
| 536 | ||
| 537 | The following process fixes these issues. | |
| 538 | To start, let's go to the `cgit` directory where we were editing our | |
| 539 | configuration file earlier. | |
| 540 | ||
| 541 | ```sh | |
| 542 | cd ~/cgit | |
| 543 | ``` | |
| 544 | ||
| 545 | In here, create two folders that will hold our syntax files: | |
| 546 | ||
| 547 | ```sh | |
| 548 | mkdir filters && mkdir filters/html-converters && cd filters | |
| 549 | ``` | |
| 550 | ||
| 551 | Next, download the default filters: | |
| 552 | ||
| 553 | ```sh | |
| 554 | curl https://git.zx2c4.com/cgit/plain/filters/about-formatting.sh > about-formatting.sh | |
| 555 | chmod 755 about-formatting.sh | |
| 556 | curl https://git.zx2c4.com/cgit/plain/filters/syntax-highlighting.py > syntax-highlighting.py | |
| 557 | chmod 755 syntax-highlighting.py | |
| 558 | ``` | |
| 559 | ||
| 560 | Finally, download the HTML conversion files you need. | |
| 561 | The example below downloads the Markdown converter: | |
| 562 | ||
| 563 | ```sh | |
| 564 | cd html-converters | |
| 565 | curl https://git.zx2c4.com/cgit/plain/filters/html-converters/md2html > md2html | |
| 566 | chmod 755 md2html | |
| 567 | ``` | |
| 568 | ||
| 569 | If you need other filters or html-converters found within [the cgit project | |
| 570 | files](https://git.zx2c4.com/cgit/tree/filters), repeat the `curl` and `chmod` | |
| 571 | process above for whichever files you need. | |
| 572 | ||
| 573 | However, formatting will not work quite yet since the Docker cgit container | |
| 574 | we're using doesn't have the formatting package installed. You can install this | |
| 575 | easily by install Python 3+ and the `pygments` package: | |
| 576 | ||
| 577 | ```sh | |
| 578 | # Enter the container's command line | |
| 579 | sudo docker exec -it cgit bash | |
| 580 | ``` | |
| 581 | ||
| 582 | ```sh | |
| 583 | # Install the necessary packages and then exit | |
| 584 | yum update -y && \ | |
| 585 | yum upgrade -y && \ | |
| 586 | yum install python3 python3-pip -y && \ | |
| 587 | pip3 install markdown pygments && \ | |
| 588 | exit | |
| 589 | ``` | |
| 590 | ||
| 591 | **You will need to enter the cgit docker container and re-run these `yum` | |
| 592 | commands every time you kill and restart the container!** | |
| 593 | ||
| 594 | If not done already, we need to add the following variables to our | |
| 595 | `cgitrc` file in order for `cgit` to know where our filtering files are: | |
| 596 | ||
| 597 | ```conf | |
| 598 | # Highlight source code with python pygments-based highlighter | |
| 599 | source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py | |
| 600 | ||
| 601 | # Format markdown, restructuredtext, manpages, text files, and html files | |
| 602 | # through the right converters | |
| 603 | about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh | |
| 604 | ``` | |
| 605 | ||
| 606 | Now you should see that syntax highlighting and README rendering to the `about` | |
| 607 | tab is fixed. | |
| 608 | ||
| 609 | ### Theming | |
| 610 | ||
| 611 | I won't go into much detail in this section, but you can fully theme your | |
| 612 | installation of `cgit` since you have access to the `cgit.css` file in your web | |
| 613 | root. This is another file you can add as a volume to the `docker-compose.yml` | |
| 614 | file if you want to edit this without entering the container's command line. | |
| 615 | ||
| 616 | ## :warning: Remember to Back Up Your Data! | |
| 617 | ||
| 618 | The last thing to note is that running services on your own equipment means that | |
| 619 | you're assuming a level of risk that exists regarding data loss, catastrophes, | |
| 620 | etc. In order to reduce the impact of any such occurrence, I suggest backing up | |
| 621 | your data regularly. | |
| 622 | ||
| 623 | Backups can be automated via `cron`, by hooking your base directory up to a | |
| 624 | cloud provider, or even setting up hooks to push all repository info to git | |
| 625 | mirrors on other git hosts. Whatever the method, make sure that your data | |
| 626 | doesn't vanish in the event that your drives or servers fail. | |
blog/2022-07-14-gnupg.org added +305
| @@ -0,0 +1,305 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-07-14 | |
| 3 | title = "GNU Privacy Guard (GPG)" | |
| 4 | description = "An overview of the GPG encryption program and its uses." | |
| 5 | draft = false | |
| 6 | +++ | |
| 7 | ||
| 8 | ## The History of GPG | |
| 9 | ||
| 10 | [GNU Privacy Guard](https://gnupg.org/), also known as GnuPG and GPG, is a free | |
| 11 | ("free" as in both speech and beer) software that fully implements the OpenPGP | |
| 12 | Message Format documented in [RFC 4880](https://www.rfc-editor.org/rfc/rfc4880). | |
| 13 | ||
| 14 | I won't go in-depth on the full history of the software in this post, but it is | |
| 15 | important to understand that GPG is not the same as PGP (Pretty Good Privacy), | |
| 16 | which is a different implementation of RFC 4880. However, GPG was designed to | |
| 17 | interoperate with PGP. | |
| 18 | ||
| 19 | GPG was originally developed in the late 1990s by [Werner | |
| 20 | Koch](https://en.wikipedia.org/wiki/Werner_Koch) and has historically been | |
| 21 | funded generously by the German government. | |
| 22 | ||
| 23 | Now that we have all the high-level info out of the way, let's dive into the | |
| 24 | different aspects of GPG and its uses. | |
| 25 | ||
| 26 | ## Encryption Algorithms | |
| 27 | ||
| 28 | GPG supports a wide range of different encryption algorithms, including | |
| 29 | public-key, cipher, hash, and compression algorithms. The support for these | |
| 30 | algorithms has grown since the adoption of the Libgcrypt library in the 2.x | |
| 31 | versions of GPG. | |
| 32 | ||
| 33 | As you will be able to see below in an example of a full key generation with the | |
| 34 | GPG command line tool, GPG recommends the following algorithms to new users: | |
| 35 | ||
| 36 | ```sh | |
| 37 | Please select what kind of key you want: | |
| 38 | (1) RSA and RSA | |
| 39 | (2) DSA and Elgamal | |
| 40 | (3) DSA (sign only) | |
| 41 | (4) RSA (sign only) | |
| 42 | (9) ECC (sign and encrypt) *default* | |
| 43 | (10) ECC (sign only) | |
| 44 | ``` | |
| 45 | ||
| 46 | I am not doing an in-depth explanation here in order to keep the focus on GPG | |
| 47 | and not encryption algorithms. If you want a deep dive into cryptography | |
| 48 | or encryption algorithms, please read my other posts: | |
| 49 | ||
| 50 | - [AES Encryption](/blog/aes-encryption/) (2018) | |
| 51 | - [Cryptography Basics](/blog/cryptography-basics/) (2020) | |
| 52 | ||
| 53 | ### Vulnerabilities | |
| 54 | ||
| 55 | As of 2022-07-14, there are a few different vulnerabilities associated with | |
| 56 | GPG or the libraries it uses: | |
| 57 | ||
| 58 | - GPG versions 1.0.2–1.2.3 contains a bug where "as soon as one | |
| 59 | (GPG-generated) ElGamal signature of an arbitrary message is released, one | |
| 60 | can recover the signer's private key in less than a second on a PC." | |
| 61 | ([Source](https://www.di.ens.fr/~pnguyen/pub_Ng04.htm)) | |
| 62 | - GPG versions prior to 1.4.2.1 contain a false positive signature | |
| 63 | verification bug. | |
| 64 | ([Source](https://lists.gnupg. | |
| 65 | org/pipermail/gnupg-announce/2006q1/000211.html)) | |
| 66 | - GPG versions prior to 1.4.2.2 cannot detect injection of unsigned data. | |
| 67 | ( | |
| 68 | [Source](https://lists.gnupg.org/pipermail/gnupg-announce/2006q1/000218.html)) | |
| 69 | - Libgcrypt, a library used by GPG, contained a bug which enabled full key | |
| 70 | recovery for RSA-1024 and some RSA-2048 keys. | |
| 71 | This was resolved in a GPG update in 2017. | |
| 72 | ([Source](https://lwn.net/Articles/727179/)) | |
| 73 | - The [ROCA Vulnerability](https://en.wikipedia.org/wiki/ROCA_vulnerability) | |
| 74 | affects RSA keys generated by YubiKey 4 tokens. | |
| 75 | ([Source](https://crocs.fi. | |
| 76 | muni.cz/_media/public/papers/nemec_roca_ccs17_preprint.pdf)) | |
| 77 | - The [SigSpoof Attack](https://en.wikipedia.org/wiki/SigSpoof) allows an | |
| 78 | attacker to spoof digital signatures. | |
| 79 | ([Source](https://arstechnica. | |
| 80 | com/information-technology/2018/06/decades-old-pgp-bug-allowed-hackers-to-spoof-just-about-anyones-signature/)) | |
| 81 | - Libgcrypt 1.9.0 contains a severe flaw related to a heap buffer overflow, | |
| 82 | fixed in Libgcrypt 1.9.1 ([Source](https://web.archive. | |
| 83 | org/web/20210221012505/https://www.theregister.com/2021/01/29/severe_libgcrypt_bug/)) | |
| 84 | ||
| 85 | ## Platforms | |
| 86 | ||
| 87 | Originally developed as a command-line program for *nix systems, GPG now has a | |
| 88 | wealth of front-end applications and libraries available for end-users. However, | |
| 89 | the most recommended programs remain the same: | |
| 90 | ||
| 91 | - [GnuPG](https://gnupg.org) for Linux (depending on distro) | |
| 92 | - [Gpg4win](https://gpg4win.org) for Windows | |
| 93 | - [GPGTools](https://gpgtools.org) for macOS | |
| 94 | ||
| 95 | ## Creating a Key Pair | |
| 96 | ||
| 97 | In order to create a GPG key pair, a user would first need to install GPG on | |
| 98 | their system. If we're assuming that the user is on Fedora Linux, they would | |
| 99 | execute the following: | |
| 100 | ||
| 101 | ```sh | |
| 102 | sudo dnf install gpg | |
| 103 | ``` | |
| 104 | ||
| 105 | Once installed, a user can create a new key pair with the following command(s): | |
| 106 | ||
| 107 | ```sh | |
| 108 | gpg --full-generate-key | |
| 109 | ``` | |
| 110 | ||
| 111 | GPG will walk the user through an interactive setup that asks for an algorithm | |
| 112 | preference, expiration date, name, and email to associate with this key. | |
| 113 | ||
| 114 | See the following example key set-up for a default key generation using the | |
| 115 | GnuPG command-line interface: | |
| 116 | ||
| 117 | ```sh | |
| 118 | gpg (GnuPG) 2.3.6; Copyright (C) 2021 Free Software Foundation, Inc. | |
| 119 | This is free software: you are free to change and redistribute it. | |
| 120 | There is NO WARRANTY, to the extent permitted by law. | |
| 121 | ||
| 122 | Please select what kind of key you want: | |
| 123 | (1) RSA and RSA | |
| 124 | (2) DSA and Elgamal | |
| 125 | (3) DSA (sign only) | |
| 126 | (4) RSA (sign only) | |
| 127 | (9) ECC (sign and encrypt) *default* | |
| 128 | (10) ECC (sign only) | |
| 129 | (14) Existing key from card | |
| 130 | Your selection? 9 | |
| 131 | Please select which elliptic curve you want: | |
| 132 | (1) Curve 25519 *default* | |
| 133 | (4) NIST P-384 | |
| 134 | Your selection? 1 | |
| 135 | Please specify how long the key should be valid. | |
| 136 | 0 = key does not expire | |
| 137 | <n> = key expires in n days | |
| 138 | <n>w = key expires in n weeks | |
| 139 | <n>m = key expires in n months | |
| 140 | <n>y = key expires in n years | |
| 141 | Key is valid for? (0) 0 | |
| 142 | Key does not expire at all | |
| 143 | Is this correct? (y/N) y | |
| 144 | ||
| 145 | GnuPG needs to construct a user ID to identify your key. | |
| 146 | ||
| 147 | Real name: John Doe | |
| 148 | Email address: johndoe@example.com | |
| 149 | Comment: test key | |
| 150 | You selected this USER-ID: | |
| 151 | "John Doe (test key) <johndoe@example.com>" | |
| 152 | ||
| 153 | Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O | |
| 154 | We need to generate a lot of random bytes. It is a good idea to perform | |
| 155 | some other action (type on the keyboard, move the mouse, utilize the | |
| 156 | disks) during the prime generation; this gives the random number | |
| 157 | generator a better chance to gain enough entropy. | |
| 158 | We need to generate a lot of random bytes. It is a good idea to perform | |
| 159 | some other action (type on the keyboard, move the mouse, utilize the | |
| 160 | disks) during the prime generation; this gives the random number | |
| 161 | generator a better chance to gain enough entropy. | |
| 162 | gpg: revocation certificate stored as 'example.rev' | |
| 163 | public and secret key created and signed. | |
| 164 | ||
| 165 | pub ed25519 2022-07-14 [SC] | |
| 166 | E955B7700FFC11EF51C2BA1FE096AACDD4C32E9C | |
| 167 | uid John Doe (test key) <johndoe@example.com> | |
| 168 | sub cv25519 2022-07-14 [E] | |
| 169 | ``` | |
| 170 | ||
| 171 | Please note that GUI apps may differ slightly from the GPG command-line | |
| 172 | interface. | |
| 173 | ||
| 174 | ## Common Usage | |
| 175 | ||
| 176 | As noted in RFC 4880, the general functions of OpenPGP are as follows: | |
| 177 | ||
| 178 | - digital signatures | |
| 179 | - encryption | |
| 180 | - compression | |
| 181 | - Radix-64 conversion | |
| 182 | - key management and certificate services | |
| 183 | ||
| 184 | From this, you can probably gather that the main use of GPG is for encrypting | |
| 185 | data and/or signing the data with a key. The purpose of encrypting data with GPG | |
| 186 | is to ensure that no one except the intended recipient(s) can access the data. | |
| 187 | ||
| 188 | Let's explore some specific GPG use-cases. | |
| 189 | ||
| 190 | ||
| 191 | ||
| 192 | One of the more popular uses of GPG is to sign and/or encrypt emails. With the | |
| 193 | use of a GPG keypair, you can encrypt a message, its subject, and even the | |
| 194 | attachments within. | |
| 195 | ||
| 196 | The first process, regarding the signing of a message without any encryption, is | |
| 197 | generally used to provide assurance that an email is truly coming from the | |
| 198 | sender that the message claims. When I send an email, and it's signed with my | |
| 199 | public key, the recipient(s) of the message can verify that the message was | |
| 200 | signed with my personal key. | |
| 201 | ||
| 202 | The second process, regarding the actual encryption of the message and its | |
| 203 | contents, works by using a combination of the sender's keys and the recipient's | |
| 204 | keys. This process may vary slightly by implementation, but it most commonly | |
| 205 | uses asymmetric cryptography, also known as public-key cryptography. In this | |
| 206 | version of encryption, the sender's private key to sign the message and a | |
| 207 | combination of the sender's keys and the recipient's public key to encrypt the | |
| 208 | message. | |
| 209 | ||
| 210 | If two people each have their own private keys and exchange their public keys, | |
| 211 | they can send encrypted messages back and forth with GPG. This is also possible | |
| 212 | with symmetric cryptography, but the process differs since there are no key | |
| 213 | pairs. | |
| 214 | ||
| 215 | Implementation of email encryption varies greatly between email clients, so you | |
| 216 | will need to reference your email client's documentation to ensure you are | |
| 217 | setting it up correctly for that specific client. | |
| 218 | ||
| 219 | ### File Encryption | |
| 220 | ||
| 221 | As noted in the section above regarding emails, GPG enables users to be able to | |
| 222 | send a message to each other if they are both set-up with GPG keys. | |
| 223 | In this example, I am going to show how a user could send a file called | |
| 224 | `example_file.txt` to another user via the recipient's email. | |
| 225 | ||
| 226 | The sender would find the file they want to send and execute the following | |
| 227 | command: | |
| 228 | ||
| 229 | ```sh | |
| 230 | gpg --encrypt --output example_file.txt.gpg --recipient \ | |
| 231 | recipient@example.com example_file.txt | |
| 232 | ``` | |
| 233 | ||
| 234 | Once received, the recipient can decrypt the file with the following command: | |
| 235 | ||
| 236 | ```sh | |
| 237 | gpg --decrypt --output example_file.txt example_file.txt.gpg | |
| 238 | ``` | |
| 239 | ||
| 240 | ### Ownership Signatures | |
| 241 | ||
| 242 | One important aspect of GPG, especially for developers, is the ability to sign | |
| 243 | data without encrypting it. For example, developers often sign code changes when | |
| 244 | they commit the changes back to a central repository, in order to display | |
| 245 | ownership of who made the changes. This allows other users to look at a code | |
| 246 | change and determine that the change was valid. | |
| 247 | ||
| 248 | In order to do this using [Git](https://git-scm.com), the developer simply needs | |
| 249 | to alter the `git commit` command to include the `-S` flag. Here's an example: | |
| 250 | ||
| 251 | ```sh | |
| 252 | git commit -S -m "my commit message" | |
| 253 | ``` | |
| 254 | ||
| 255 | As an expansion of the example above, Git users can configure their environment | |
| 256 | with a default key to use by adding their GPG signature: | |
| 257 | ||
| 258 | ```sh | |
| 259 | git config --global user.signingkey XXXXXXXXXXXXXXXX | |
| 260 | ``` | |
| 261 | ||
| 262 | If you're not sure what your signature is, you can find it titled `sig` in the | |
| 263 | output of this command: | |
| 264 | ||
| 265 | ```sh | |
| 266 | gpg --list-signatures | |
| 267 | ``` | |
| 268 | ||
| 269 | ### File Integrity | |
| 270 | ||
| 271 | When a person generates a signature for data, they are allowing users the | |
| 272 | ability to verify the signature on that data in the future to ensure the data | |
| 273 | has not been corrupted. This is most common with software applications hosted on | |
| 274 | the internet - developers provide signatures so that users can verify a website | |
| 275 | was not hijacked and download links replaced with dangerous software. | |
| 276 | ||
| 277 | In order to verify signed data, a user needs to have: | |
| 278 | ||
| 279 | 1. The signed data | |
| 280 | 2. A signature file | |
| 281 | 3. The public GPG key of the signer | |
| 282 | ||
| 283 | Once the signer's public key is imported on the user's system, and they have | |
| 284 | the data and signature, they can verify the data with the following commands: | |
| 285 | ||
| 286 | ```sh | |
| 287 | # If the signature is attached to the data | |
| 288 | gpg --verify [signature-file] | |
| 289 | ||
| 290 | # If the signature is detached as a separate file from the data | |
| 291 | gpg --verify [signature-file] [original-file] | |
| 292 | ``` | |
| 293 | ||
| 294 | ## Finding Public Keys | |
| 295 | ||
| 296 | In order to use GPG with others, a user needs to know the other user(s) keys. | |
| 297 | This is easy to do if the user knows the other user(s) in person, but may be | |
| 298 | hard if the relationship is strictly digital. Luckily, there are a few options. | |
| 299 | The first option is to look at a user's web page or social pages if they have | |
| 300 | them. | |
| 301 | ||
| 302 | Otherwise, the best option is to use a keyserver, such as: | |
| 303 | ||
| 304 | - [pgp.mit.edu](https://pgp.mit.edu) | |
| 305 | - [keys.openpgp.org](https://keys.openpgp.org) | |
blog/2022-07-25-curseradio.org added +99
| @@ -0,0 +1,99 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-07-25 | |
| 3 | title = "CurseRadio: Listening to the Radio on the Command Line" | |
| 4 | description = "A quick and easy installation for CurseRadio on Fedora Linux." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | While exploring some interesting Linux applications, I stumbled across | |
| 10 | [curseradio](https://github.com/chronitis/curseradio), a command-line radio | |
| 11 | player based on Python. | |
| 12 | ||
| 13 | This application is fantastic and incredibly easy to install, so I wanted to | |
| 14 | dedicate a post today to this app. Let's look at the features within the app and | |
| 15 | then walk through the installation process I took to get `curseradio` working. | |
| 16 | ||
| 17 | ## Features | |
| 18 | ||
| 19 |  | |
| 20 | ||
| 21 | The radio player itself is quite minimal. As you can see in the screenshot | |
| 22 | above, it contains a simple plaintext list of all available categories, which | |
| 23 | can be broken down further and further. In addition, radio shows are available | |
| 24 | for listening, alongside regular radio stations. | |
| 25 | ||
| 26 | For example, the `Sports` > `Pro Basketball` > `Shows` category contains a | |
| 27 | number of specific shows related to Professional Basketball. | |
| 28 | ||
| 29 | Aside from being able to play any of the listed stations/shows, you can make | |
| 30 | a channel your favorite by pressing `f`. | |
| 31 | It will now show up at the top of the radio player in the `Favourites` category. | |
| 32 | ||
| 33 | ### Commands/Shortcuts | |
| 34 | ||
| 35 | | Key(s) | Command | | |
| 36 | |:----------:|:-------------------------------:| | |
| 37 | | ↑, ↓ | navigate | | |
| 38 | | PgUp, PgDn | navigate quickly | | |
| 39 | | Home, End | to top/bottom | | |
| 40 | | Enter | open/close folders, play stream | | |
| 41 | | k | stop playing stream | | |
| 42 | | q | quit | | |
| 43 | | f | toggle favourite | | |
| 44 | ||
| 45 | ## Installation | |
| 46 | ||
| 47 | ### Dependencies | |
| 48 | ||
| 49 | Before installing `curseradio`, a handful of system and Python packages are | |
| 50 | required. To get started, install `python3`, `pip3`, and `mpv` on your system. | |
| 51 | In this example, I'm using Fedora Linux, which uses the `dnf` package manager. | |
| 52 | You may need to adjust this if you're using a different system. | |
| 53 | ||
| 54 | ```sh | |
| 55 | sudo dnf install python3 pip3 mpv | |
| 56 | ``` | |
| 57 | ||
| 58 | Next, use `pip3` to install `requests`, `xdg`, and `lxml`: | |
| 59 | ||
| 60 | ```sh | |
| 61 | pip3 install requests xdg lxml | |
| 62 | ``` | |
| 63 | ||
| 64 | ### Repository Source Installation | |
| 65 | ||
| 66 | Once all the dependencies are installed, we can clone the source code and enter | |
| 67 | that directory: | |
| 68 | ||
| 69 | ```sh | |
| 70 | git clone https://github.com/chronitis/curseradio && cd curseradio | |
| 71 | ``` | |
| 72 | ||
| 73 | Once you're within the `curseradio` directory, you can install the application | |
| 74 | with the provided `setup.py` script. | |
| 75 | ||
| 76 | ```sh | |
| 77 | sudo python3 setup.py install | |
| 78 | ``` | |
| 79 | ||
| 80 | In my case, I ran into a few errors and needed to create the folders that | |
| 81 | curseradio wanted to use for its installation. If you don't get any errors, you | |
| 82 | can skip this and run the app. | |
| 83 | ||
| 84 | ```sh | |
| 85 | sudo mkdir /usr/local/lib/python3.10/ | |
| 86 | sudo mkdir /usr/local/lib/python3.10/site-packages/ | |
| 87 | ``` | |
| 88 | ||
| 89 | ```sh | |
| 90 | sudo chown -R $USER:$USER /usr/local/lib/python3.10/ | |
| 91 | ``` | |
| 92 | ||
| 93 | ## Run the Application | |
| 94 | ||
| 95 | Once fully installed without errors, you can run the application! | |
| 96 | ||
| 97 | ```sh | |
| 98 | python3 /usr/local/bin/curseradio | |
| 99 | ``` | |
blog/2022-07-30-flac-to-opus.org added +174
| @@ -0,0 +1,174 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-07-30 | |
| 3 | title = "Recursive Command-Line FLAC to Opus Conversion" | |
| 4 | description = "A retrospective on how I recursively converting my FLAC file music library to OPUS with opusenc." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Converting FLAC to OPUS | |
| 8 | ||
| 9 | I am currently rebuilding my music library from scratch so that I can | |
| 10 | effectively archive all the music I own in the [FLAC file | |
| 11 | format](https://en.wikipedia.org/wiki/FLAC), a lossless audio codec. | |
| 12 | ||
| 13 | However, streaming FLAC files outside the home can be difficult due to the | |
| 14 | size of the files, especially if you're using a weak connection. | |
| 15 | ||
| 16 | So, in order to archive the music in a lossless format and still be able to | |
| 17 | stream it easily, I opted to create a copy of my FLAC files in the [Opus audio | |
| 18 | codec](https://en.wikipedia.org/wiki/Opus_(audio_format)). This allows me to | |
| 19 | archive a quality, lossless version of the music and then point my streaming | |
| 20 | service to the smaller, stream-ready version. | |
| 21 | ||
| 22 | ### Dependencies | |
| 23 | ||
| 24 | The process I follow utilizes the `opus-tools` package in Ubuntu. Before | |
| 25 | proceeding, install the package: | |
| 26 | ||
| 27 | ```sh | |
| 28 | sudo apt install opus-tools | |
| 29 | ``` | |
| 30 | ||
| 31 | If you want to use a different conversion method, such as `ffmpeg` or `avconv`, | |
| 32 | simply install that package instead. | |
| 33 | ||
| 34 | ### Conversion Process | |
| 35 | ||
| 36 | The script I'm using is stored in my home directory, but feel free to create it | |
| 37 | wherever you want. It does not need to be in the same directory as your music | |
| 38 | files. | |
| 39 | ||
| 40 | ```sh | |
| 41 | cd ~ && nano transform.sh | |
| 42 | ``` | |
| 43 | ||
| 44 | Once you have your new bash script opened in an editor, go ahead and paste the | |
| 45 | following logic into the script. | |
| 46 | ||
| 47 | You **MUST** edit the following variables in order for it to work: | |
| 48 | ||
| 49 | - `source`: The source directory where your FLAC files are stored. | |
| 50 | - `dest`: The destination directory where you want the resulting Opus files to | |
| 51 | be stored. | |
| 52 | ||
| 53 | You **MAY** want to edit the following variables to suit your needs: | |
| 54 | ||
| 55 | - `filename`: If you are converting to a file format other than Opus, you'll | |
| 56 | need to edit this so that your resulting files have the correct filename | |
| 57 | extension. | |
| 58 | - `reldir`: This variable can be edited to strip out more leading directories in | |
| 59 | the file path. As you'll see later, I ignore this for now and simply clean it up | |
| 60 | afterward. | |
| 61 | - `opusenc`: This is the actual conversion process. You may want to edit the | |
| 62 | bitrate to suit your needs. I set mine at 128 but some prefer 160 or higher. | |
| 63 | ||
| 64 | ```sh | |
| 65 | #!/bin/bash | |
| 66 | ## - The IFS takes care of spaces in file and dirnames | |
| 67 | ## - your folders may vary | |
| 68 | ## - what you mount to the folders does not matter | |
| 69 | ## - in RELDIR, the f5 most likely MUST be edited, | |
| 70 | ## since its responsible, how many leading directories | |
| 71 | ## will be removed from the directory structure in order | |
| 72 | ## to append that exact path to the outfile | |
| 73 | ## - the commented echos are still in place in order to give | |
| 74 | ## you the variables for testing, before running. | |
| 75 | ||
| 76 | IFS=$'\n' | |
| 77 | ||
| 78 | ## the paths given here contain the directory structure that I want to keep | |
| 79 | ## source=/mnt/music/archives/ARTIST/ALBUM/FLACFILE.flac | |
| 80 | ## local=/mnt/music/library/ARTIST/ALBUM/OPUSFILE.opus | |
| 81 | ||
| 82 | source=/mnt/music/archives | |
| 83 | dest=/mnt/music/library | |
| 84 | ||
| 85 | for i in $(find $source -type f -iname '*.flac' ); | |
| 86 | do | |
| 87 | ## SET VARIABLES for PATHS and FILENAMES | |
| 88 | fullfile=$i | |
| 89 | filename="${i##*/}" | |
| 90 | filename="${filename%.*}.opus" | |
| 91 | fulldir=$(dirname "${i}") | |
| 92 | reldir="$(echo $fulldir | cut -d'/' -f5-)" | |
| 93 | reldir=${reldir//flac} | |
| 94 | outdir="$dest/$reldir" | |
| 95 | outfile="$outdir/$filename" | |
| 96 | ||
| 97 | # is that working? | |
| 98 | # outfile='$local/""$(echo $(dirname "${i}") | cut -d'/' -f5-)"//flac"/"${i##*/}"' | |
| 99 | # echo 'output file: ' "$outfile" | |
| 100 | ||
| 101 | ## SHOW ME THE CONTENTS of the VARIABLES | |
| 102 | # echo 'File found:' "$i" | |
| 103 | # echo 'Relative dir: ' "$reldir" | |
| 104 | # echo 'directory will be created: ' "$outdir" | |
| 105 | # echo 'Filename: ' "$filename" | |
| 106 | # echo 'FileExt: ' "$extension" | |
| 107 | # echo 'output file: ' "$outfile" | |
| 108 | ||
| 109 | echo "\n\n" | |
| 110 | ||
| 111 | ## CREATE Output Folders | |
| 112 | mkdir -p "$outdir" | |
| 113 | ||
| 114 | ## RUN | |
| 115 | # ffmpeg and avconv are alternative options if opusenc isn't adequate | |
| 116 | opusenc --vbr --bitrate 128 --date "$DATE" \ | |
| 117 | --title "$TITLE" --artist "$ARTIST" --album "$ALBUM" --genre "$GENRE" \ | |
| 118 | --comment "ALBUMARTIST=$ALBUMARTIST" --comment "DISCNUMBER=$DISCNUMBER" \ | |
| 119 | --comment "TRACKNUMBER=$TRACKNUMBER" --comment "TRACKTOTAL=$TRACKTOTAL" \ | |
| 120 | --comment "LYRICS=$LYRICS" "$fullfile" "$outfile" | |
| 121 | ||
| 122 | ||
| 123 | ## just for testing | |
| 124 | # sleep 1 | |
| 125 | done | |
| 126 | ``` | |
| 127 | ||
| 128 | Once you're done, simply save the file and exit your editor. Don't forget to | |
| 129 | enable execution of the script: | |
| 130 | ||
| 131 | ```sh | |
| 132 | chmod +x transform.sh | |
| 133 | ``` | |
| 134 | ||
| 135 | Finally, you may now run the script: | |
| 136 | ||
| 137 | ```sh | |
| 138 | ./transform.sh | |
| 139 | ``` | |
| 140 | ||
| 141 | If you used `opusenc`, you'll see the conversions happen within the terminal as | |
| 142 | it progresses. | |
| 143 | You will also see variables printed if you uncommented any of the bash | |
| 144 | script's comments. | |
| 145 | ||
| 146 | ### Cleanup | |
| 147 | ||
| 148 | As I noted above, I didn't customize my `reldir` variable in the script, which | |
| 149 | caused my output directory to be `/mnt/music/library/archives` instead of | |
| 150 | `/mnt/music/library`. So, I moved the output up one level and deleted the | |
| 151 | accidental directory. | |
| 152 | ||
| 153 | ```sh | |
| 154 | cd /mnt/music/library | |
| 155 | mv archives/* . | |
| 156 | rm -rf archives | |
| 157 | ``` | |
| 158 | ||
| 159 | ### Check the Resulting Size | |
| 160 | ||
| 161 | If you want to see what kind of file size savings you've gained, you can always | |
| 162 | use the `du` command to check: | |
| 163 | ||
| 164 | ```sh | |
| 165 | cd /mnt/music | |
| 166 | du -h --max-depth=1 . | |
| 167 | ``` | |
| 168 | ||
| 169 | In my case, my small library went from 78GB to 6.3GB! | |
| 170 | ||
| 171 | ```txt | |
| 172 | 78G ./archives | |
| 173 | 6.3G ./library | |
| 174 | ``` | |
blog/2022-07-31-bash-it.org added +237
| @@ -0,0 +1,237 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-07-31 | |
| 3 | title = "Upgrade Bash with Bash-It & Ble.sh" | |
| 4 | description = "In this post, I walk through the Bash-It and Ble.sh projects, their features, and how they help users power-up their bash shell." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Bash | |
| 8 | ||
| 9 | For those who are not familiar, | |
| 10 | [Bash](https://en.wikipedia.org/wiki/Bash_(Unix_shell)) is a Unix shell that is | |
| 11 | used as the default login shell for most Linux distributions. This shell and | |
| 12 | command processor should be familiar if you've used Linux (or older version of | |
| 13 | macOS) before. | |
| 14 | ||
| 15 | However, bash is not the only option. | |
| 16 | There are numerous other shells that exist. | |
| 17 | Here are some popular examples: | |
| 18 | ||
| 19 | - [zsh](https://en.wikipedia.org/wiki/Z_shell) | |
| 20 | - [fish](https://en.wikipedia.org/wiki/Fish_(Unix_shell)) | |
| 21 | - [oksh](https://github.com/ibara/oksh) | |
| 22 | - [mksh](https://wiki.gentoo.org/wiki/Mksh) | |
| 23 | - [dash](https://en.wikipedia.org/wiki/Debian_Almquist_shell) | |
| 24 | ||
| 25 | While each shell has its differences, bash is POSIX compliant and the default | |
| 26 | for many Linux users. Because of this, I am going to explore a program called | |
| 27 | `bash-it` below that helps bash users increase the utility of their shell | |
| 28 | without installing a completely new shell. | |
| 29 | ||
| 30 | ### Installation | |
| 31 | ||
| 32 | First, if bash is not already installed on your system, you can [download bash | |
| 33 | from GNU](https://www.gnu.org/software/bash/) or use your package manager to | |
| 34 | install it. | |
| 35 | ||
| 36 | For example, this is how you can install bash on Fedora Linux: | |
| 37 | ||
| 38 | ```sh | |
| 39 | sudo dnf install bash | |
| 40 | ``` | |
| 41 | ||
| 42 | If you are not using bash as your default shell, use the `chsh` command to | |
| 43 | change your shell: | |
| 44 | ||
| 45 | ```sh | |
| 46 | chsh | |
| 47 | ``` | |
| 48 | ||
| 49 | You should see a prompt like the one below. If the brackets (`[]`) contain | |
| 50 | `bash` already, you're done, and you can simply continue by hitting the Enter | |
| 51 | key. | |
| 52 | ||
| 53 | If the brackets contain another shell path (e.g. `/usr/bin/zsh`), enter the path | |
| 54 | to the bash program on your system (it's most likely located at | |
| 55 | `/usr/bin/bash`). | |
| 56 | ||
| 57 | ```sh | |
| 58 | Changing shell for <user>. | |
| 59 | New shell [/usr/bin/bash]: | |
| 60 | ``` | |
| 61 | ||
| 62 | You must log out or restart the machine in order for the login shell to be | |
| 63 | refreshed. You can do it now or wait until you're finished customizing the | |
| 64 | shell. | |
| 65 | ||
| 66 | ```sh | |
| 67 | sudo reboot now | |
| 68 | ``` | |
| 69 | ||
| 70 | ## Bash-it | |
| 71 | ||
| 72 | As noted on the [Bash-it](https://github.com/Bash-it/bash-it) repository: | |
| 73 | ||
| 74 | > Bash-it is a collection of community Bash commands and scripts for Bash 3.2+. | |
| 75 | > (And a shameless ripoff of oh-my-zsh 😃) | |
| 76 | ||
| 77 | Bash-it makes it easy to install plugins, set up aliases for common commands, | |
| 78 | and easily change the visual theme of your shell. | |
| 79 | ||
| 80 | ### Installation | |
| 81 | ||
| 82 | To install the framework, simply copy the repository files and use the | |
| 83 | `install.sh` script provided. If you want, you can (and should!) inspect the | |
| 84 | contents of the installation script before you run it. | |
| 85 | ||
| 86 | ```sh | |
| 87 | git clone --depth=1 https://github.com/Bash-it/bash-it.git ~/.bash_it | |
| 88 | ~/.bash_it/install.sh | |
| 89 | ``` | |
| 90 | ||
| 91 | If you didn't restart your session after making bash the default, and are | |
| 92 | currently working within another shell, be sure to enter a bash session before | |
| 93 | using `bash-it`: | |
| 94 | ||
| 95 | ```sh | |
| 96 | bash | |
| 97 | ``` | |
| 98 | ||
| 99 | ### Aliases | |
| 100 | ||
| 101 | Bash-it contains a number of aliases for common commands to help improve | |
| 102 | efficiency in the terminal. To list all available options, use the following | |
| 103 | command: | |
| 104 | ||
| 105 | ```sh | |
| 106 | bash-it show aliases | |
| 107 | ``` | |
| 108 | ||
| 109 | This will provide you a list that looks like the following text block. | |
| 110 | Within this screen, you will be able to see all available options and which | |
| 111 | ones are currently enabled. | |
| 112 | ||
| 113 | ```txt | |
| 114 | Alias Enabled? Description | |
| 115 | ag [ ] the silver searcher (ag) aliases | |
| 116 | ansible [ ] ansible abbreviations | |
| 117 | apt [ ] Apt and dpkg aliases for Ubuntu and Debian distros. | |
| 118 | atom [ ] Atom.io editor abbreviations | |
| 119 | bash-it [ ] Aliases for the bash-it command (these aliases are automatically included with the "general" aliases) | |
| 120 | bolt [ ] puppet bolt aliases | |
| 121 | bundler [ ] ruby bundler | |
| 122 | clipboard [ ] xclip shortcuts | |
| 123 | composer [ ] common composer abbreviations | |
| 124 | curl [x] Curl aliases for convenience. | |
| 125 | ... | |
| 126 | ``` | |
| 127 | ||
| 128 | To enable an alias, do: | |
| 129 | ||
| 130 | ```sh | |
| 131 | bash-it enable alias <alias name> [alias name]... -or- $ bash-it enable alias all | |
| 132 | ``` | |
| 133 | ||
| 134 | To disable an alias, do: | |
| 135 | ||
| 136 | ```sh | |
| 137 | bash-it disable alias <alias name> [alias name]... -or- $ bash-it disable alias all | |
| 138 | ``` | |
| 139 | ||
| 140 | ### Plugins | |
| 141 | ||
| 142 | Similar to aliases, plugins are available with bash-it. You can find a complete | |
| 143 | list of plugins in the same way as aliases. Simply execute the following: | |
| 144 | ||
| 145 | ```sh | |
| 146 | bash-it show plugins | |
| 147 | ``` | |
| 148 | ||
| 149 | You will see the following output showing enabled and disabled plugins: | |
| 150 | ||
| 151 | ```txt | |
| 152 | Plugin Enabled? Description | |
| 153 | alias-completion [ ] | |
| 154 | autojump [ ] Autojump configuration, see https://github.com/wting/autojump for more details | |
| 155 | aws [ ] AWS helper functions | |
| 156 | base [x] miscellaneous tools | |
| 157 | basher [ ] initializes basher, the shell package manager | |
| 158 | battery [x] display info about your battery charge level | |
| 159 | blesh [ ] load ble.sh, the Bash line editor! | |
| 160 | boot2docker [ ] Helpers to get Docker setup correctly for boot2docker | |
| 161 | browser [ ] render commandline output in your browser | |
| 162 | ``` | |
| 163 | ||
| 164 | To enable a plugin, do: | |
| 165 | ||
| 166 | ```sh | |
| 167 | bash-it enable plugin <plugin name> [plugin name]... -or- $ bash-it enable plugin all | |
| 168 | ``` | |
| 169 | ||
| 170 | To disable a plugin, do: | |
| 171 | ||
| 172 | ```sh | |
| 173 | bash-it disable plugin <plugin name> [plugin name]... -or- $ bash-it disable plugin all | |
| 174 | ``` | |
| 175 | ||
| 176 | ### Themes | |
| 177 | ||
| 178 | There are quite a few pre-defined | |
| 179 | [themes](https://bash-it.readthedocs.io/en/latest/themes-list/#list-of-themes) | |
| 180 | available with bash-it. | |
| 181 | ||
| 182 | To list all themes: | |
| 183 | ||
| 184 | ```sh | |
| 185 | ls ~/.bash_it/themes/ | |
| 186 | ``` | |
| 187 | ||
| 188 | To use a new theme, you'll need to edit `.bashrc` and alter the `BASH_IT_THEME` | |
| 189 | variable to your desired theme. For example, I am using the `zork` theme. | |
| 190 | ||
| 191 | ```sh | |
| 192 | nano ~/.bashrc | |
| 193 | ``` | |
| 194 | ||
| 195 | ```sh | |
| 196 | export BASH_IT_THEME='zork' | |
| 197 | ``` | |
| 198 | ||
| 199 | Once you save your changes, you just need to exit your terminal and create a new | |
| 200 | one in order to see your changes to the `.bashrc` file. You can also `source` | |
| 201 | the file to see changes, but I recommend starting a completely new shell | |
| 202 | instead. | |
| 203 | ||
| 204 | ## ble.sh | |
| 205 | ||
| 206 | One big feature I was missing in Bash that both `zsh` and `fish` have is an | |
| 207 | autosuggestion feature. To explain: as you type, an autosuggestion feature in | |
| 208 | the shell will offer suggestions in a lighter font color beyond the characters | |
| 209 | already typed. Once you see the command you want, you can click the right arrow | |
| 210 | and have the shell auto-complete that line for you. | |
| 211 | ||
| 212 | Luckily, the [Bash Line Editor](https://github.com/akinomyoga/ble.sh) (ble.sh) | |
| 213 | exists! This program provides a wonderful autosuggestions feature perfectly, | |
| 214 | among other features that I haven't tested yet. | |
| 215 | ||
| 216 | In order to install ble.sh, execute the following: | |
| 217 | ||
| 218 | ```sh | |
| 219 | git clone --recursive https://github.com/akinomyoga/ble.sh.git | |
| 220 | make -C ble.sh install PREFIX=~/.local | |
| 221 | echo 'source ~/.local/share/blesh/ble.sh' >> ~/.bashrc | |
| 222 | ``` | |
| 223 | ||
| 224 | Again, exit the terminal and open a new one in order to see the newly-configured | |
| 225 | shell. | |
| 226 | ||
| 227 | ## Restart the Session | |
| 228 | ||
| 229 | Finally, as mentioned above, you'll need to restart the session to ensure that | |
| 230 | your user is using bash by default. | |
| 231 | ||
| 232 | You will also need to exit and re-open a shell (e.g., terminal or terminal tab) | |
| 233 | any time you make changes to the `.bashrc` file. | |
| 234 | ||
| 235 | ```sh | |
| 236 | sudo reboot now | |
| 237 | ``` | |
blog/2022-08-31-privacy.com-changes.org added +92
| @@ -0,0 +1,92 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-08-31 | |
| 3 | title = "Concerning Changes on Privacy.com" | |
| 4 | description = "A quick description of the recent changes of Privacy.com toward a credit card model." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Privacy.com Changes Their Terms | |
| 8 | ||
| 9 | Recently, Privacy.com reached out to their customers regarding a change in their | |
| 10 | terms of use. Further, all customers are required to agree to the changes in | |
| 11 | order to continue using their accounts. | |
| 12 | ||
| 13 | [You can view the new cardholder agreement | |
| 14 | here](https://privacy.com/commercial-cardholder-agreement). | |
| 15 | ||
| 16 | When you log in, you'll be greeted with a pop-up window asking you to review | |
| 17 | and agree to the new terms of use. | |
| 18 | You will also not be able to open any new cards until the terms are agreed to. | |
| 19 | ||
| 20 | ### Changing from a "Prepaid Debit" Model to a "Charge Card" Model | |
| 21 | ||
| 22 | The actual content of the changes is interesting. While the historical model of | |
| 23 | using Privacy.com was akin to prepaid debit cards, the new model is very similar | |
| 24 | to a credit card (they use the term "charge card"). | |
| 25 | ||
| 26 | I have used Privacy.com for 1-2 years, and the process was always that you | |
| 27 | would create a single-use or merchant-locked card. | |
| 28 | This card could be used for any dollar limit you set and would immediately | |
| 29 | draw the funds from whatever funding source you connected, e.g. PayPal | |
| 30 | account or a bank account. | |
| 31 | ||
| 32 | The benefit this service provides with a premium account is masking the merchant | |
| 33 | names from your funding source. If you have a concern that your bank account | |
| 34 | uses merchant data from your account, you could direct all charges through | |
| 35 | Privacy.com and set the merchant as one of their pre-set options, such as | |
| 36 | "Smiley's Corner Store" or "NSA Gift Shop." | |
| 37 | ||
| 38 | The new model still works with a bank account as a funding source, but the model | |
| 39 | is changed so that you get a "line of credit" set according to a 14-day billing | |
| 40 | cycle. It seems that Privacy.com will now allow charges to be incurred without | |
| 41 | being immediately paid. | |
| 42 | ||
| 43 | ### Daily Payments and Available Credit | |
| 44 | ||
| 45 | Instead of paying as charges are incurred, you must make a "Daily Payment" and | |
| 46 | your "Available Credit" will be locked until you make that payment. There are | |
| 47 | also "End of Billing Cycle Payments" that are assigned a due date. | |
| 48 | ||
| 49 | Further, Privacy.com will decline charges that would cause you to exceed your | |
| 50 | Available Credit or Credit Limit. | |
| 51 | ||
| 52 | One particular interesting section states the following: | |
| 53 | ||
| 54 | > YOUR OBLIGATION TO PAY US BACK FOR ALL CARD TRANSACTIONS AND OTHER OBLIGATIONS | |
| 55 | > YOU INCUR IS SECURED BY THE SECURED ACCOUNT. IF YOU DO NOT PAY US BACK FOR ANY | |
| 56 | > AMOUNT YOU OWE US WHEN YOUR PAYMENTS ARE DUE, WE WILL EXERCISE OUR INTEREST | |
| 57 | > AND DEBIT THE SECURED ACCOUNT, AND YOU WILL LOSE THE MONEY IN THE SECURED | |
| 58 | > ACCOUNT. SEE SECTION 8: SECURITY AGREEMENT FOR MORE INFORMATION. | |
| 59 | ||
| 60 | ### Personal Information | |
| 61 | ||
| 62 | Now that Privacy.com is more of a financial institution, they are obligated to | |
| 63 | comply with the [know your | |
| 64 | customer](https://en.wikipedia.org/wiki/Know_your_customer) guidelines/laws. | |
| 65 | ||
| 66 | I did not proceed with the change to my Privacy.com account, but I have heard | |
| 67 | from some peers that the changes require more personal information to be | |
| 68 | submitted, such as SSN. I am not aware of all new personal information required | |
| 69 | or if the funding source is now required to only be a bank account. | |
| 70 | ||
| 71 | ### Fees | |
| 72 | ||
| 73 | Luckily, the fees section did not change much. The subscription fees for a | |
| 74 | premium account are still the only fees. | |
| 75 | ||
| 76 | ## My Thoughts | |
| 77 | ||
| 78 | Personally, I wiped my personal information from my account and then permanently | |
| 79 | deleted it when I heard about these changes. I have no interest in yet another | |
| 80 | method of credit lending offered by private companies. While I accepted that | |
| 81 | they would have access to my bank account information for the purposes of paying | |
| 82 | off my prepaid debit payments, I have no interest in incurring charges that will | |
| 83 | need to be paid back at a later date. I also have no interest in submitting | |
| 84 | personal information to Privacy.com. | |
| 85 | ||
| 86 | This type of change toward a "buy it now, pay us later" model is concerning, | |
| 87 | and I will be watching Privacy.com to see if they further their interests in | |
| 88 | the credit model as time goes on. | |
| 89 | ||
| 90 | Could we see them start charging interest, fees, etc.? I'm not sure, but | |
| 91 | this change does not inspire confidence in their mission as a | |
| 92 | privacy-focused company. | |
blog/2022-09-17-serenity-os.org added +113
| @@ -0,0 +1,113 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-09-17 | |
| 3 | title = "Serenity OS: Testing Out a Unique System" | |
| 4 | description = "Walking through my exploration of Serenity OS, a unique 90s-esque system." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | [SerenityOS](https://serenityos.org) is a unique operating system (OS) that I | |
| 10 | have seen pop up in my news feed a few times over the last few years, but I have | |
| 11 | never had time to test it out until now. | |
| 12 | ||
| 13 | Testing out this system brought back fond memories of yellowed, | |
| 14 | modem-screeching, 100-pound computers that brought so many fond memories to my | |
| 15 | youth. | |
| 16 | ||
| 17 | Per their website: | |
| 18 | ||
| 19 | > A graphical Unix-like operating system for desktop computers! | |
| 20 | > | |
| 21 | > SerenityOS is a love letter to '90s user interfaces with a custom Unix-like | |
| 22 | > core. | |
| 23 | > It flatters with sincerity by stealing beautiful ideas from various | |
| 24 | > other systems. | |
| 25 | > | |
| 26 | > Roughly speaking, the goal is a marriage between the aesthetic of late-1990s | |
| 27 | > productivity software and the power-user accessibility of late-2000s *nix. | |
| 28 | > | |
| 29 | > This is a system by us, for us, based on the things we like. | |
| 30 | ||
| 31 | ## Building | |
| 32 | ||
| 33 | Your first question may be "Where's the iso?" and the answer is... there are | |
| 34 | none. SerenityOS does not provide pre-built images for testing. You must build | |
| 35 | the images yourself. This seems intentionally to limit participation to only | |
| 36 | those who are truly interested enough to learn how to build the OS. | |
| 37 | ||
| 38 | ### Clone | |
| 39 | ||
| 40 | In order to get started, you'll need to clone the source repository: | |
| 41 | ||
| 42 | ```sh | |
| 43 | git clone https://github.com/SerenityOS/serenity && cd serenity | |
| 44 | ``` | |
| 45 | ||
| 46 | ### Build | |
| 47 | ||
| 48 | Note that I followed the [Build | |
| 49 | Instructions](https://github.com/SerenityOS/serenity/blob/master/Documentation/BuildInstructions.md) | |
| 50 | in the SerenityOS repository as of commit | |
| 51 | `660d2b53b1206e868d5470eee80b5e62d7e30da7`. | |
| 52 | Things may have changed since my installation, and you should double-check | |
| 53 | the instructions first. | |
| 54 | ||
| 55 | Regardless, I want to repeat my steps here to illustrate any errors or differing | |
| 56 | commands I needed to run in order to build and run SerenityOS. | |
| 57 | ||
| 58 | Since I am running Fedora, I needed to install these packages in order to build | |
| 59 | the OS images: | |
| 60 | ||
| 61 | ```sh | |
| 62 | sudo dnf install texinfo binutils-devel curl cmake mpfr-devel libmpc-devel gmp-devel e2fsprogs ninja-build patch ccache rsync @"C Development Tools and Libraries" @Virtualization | |
| 63 | ``` | |
| 64 | ||
| 65 | Next, make sure you're inside the `serenity` directory created earlier | |
| 66 | during the git cloning process and process to build the toolchain: | |
| 67 | ||
| 68 | ```sh | |
| 69 | Meta/serenity.sh rebuild-toolchain | |
| 70 | ``` | |
| 71 | ||
| 72 | Once the toolchain is built, you can Build and run the OS! | |
| 73 | ||
| 74 | ```sh | |
| 75 | Meta/serenity.sh run | |
| 76 | ``` | |
| 77 | ||
| 78 | After this process is completed, the image should run automatically and launch. | |
| 79 | ||
| 80 | ## Issues | |
| 81 | ||
| 82 | I played around in SerenityOS for an hour or two in order to see what I could do | |
| 83 | and had a lot of fun with it. The only issue I ran into was a lack of working | |
| 84 | internet. I didn't try very hard, but I could tell that the main network link | |
| 85 | wasn't connecting to my Fedora host properly. | |
| 86 | ||
| 87 | ## Screenshots | |
| 88 | ||
| 89 | The initial launch of the image displays the SerenityOS desktop, with a simple | |
| 90 | terminal already launched: | |
| 91 | ||
| 92 |  | |
| 93 | ||
| 94 | Here you can see the Fire application (literally just shows fire burning), a | |
| 95 | browser with the local Serenity Browser page loaded, and a text editor. | |
| 96 | ||
| 97 |  | |
| 98 | ||
| 99 | I also poked around the system utilities and found most tools you'd expect to | |
| 100 | find within a standard desktop. | |
| 101 | ||
| 102 |  | |
| 103 | ||
| 104 | Lastly, I noted that the default desktop contains numerous pre-defined themes to | |
| 105 | choose from. This is a small piece, but it's actually wonderful to see desktop | |
| 106 | developers consider theming directly out of the box rather than using an | |
| 107 | addon-based mentality. | |
| 108 | ||
| 109 |  | |
| 110 | ||
| 111 | I didn't take a screenshot of the other pre-installed games, but I did spend | |
| 112 | nearly 30 minutes playing Solitaire before remembering that I was supposed to be | |
| 113 | writing a post about the OS. | |
blog/2022-09-21-graphene-os.org added +160
| @@ -0,0 +1,160 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-09-21 | |
| 3 | title = "Installing Graphene OS on the Pixel 6 Pro" | |
| 4 | description = "A walkthrough tutorial on how to install GrapheneOS on the Pixel 6 Pro with Fedora Linux." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Introduction | |
| 8 | ||
| 9 | After using iOS for a couple of years, I finally took the plunge and | |
| 10 | purchased a Pixel 6 Pro in order to test and use [GrapheneOS] | |
| 11 | (https://grapheneos.org). | |
| 12 | ||
| 13 | The installation process was rather quick once you have the tools and files you | |
| 14 | need. Overall, it can be done in just a few minutes. | |
| 15 | ||
| 16 | ## Gathering Tools & Files | |
| 17 | ||
| 18 | ### Android Tools | |
| 19 | ||
| 20 | First, in order to interact with the device, we will need the [Android | |
| 21 | platform | |
| 22 | tools](https://developer.android.com/studio/releases/platform-tools.html). Find | |
| 23 | the Linux download and save the ZIP folder to your preferred location. | |
| 24 | ||
| 25 | Once we've downloaded the files, we will need to unzip them, enter the | |
| 26 | directory, and move the necessary executables to a central location, such as | |
| 27 | `/usr/bin/`. For this installation, we only need the `fastboot` and `adb` | |
| 28 | executables. | |
| 29 | ||
| 30 | ```sh | |
| 31 | cd ~/Downloads | |
| 32 | ``` | |
| 33 | ||
| 34 | ```sh | |
| 35 | unzip platform-tools_r33.0.3-linux.zip | |
| 36 | cd platform-tools | |
| 37 | sudo mv fastboot /usr/bin/ | |
| 38 | sudo mv adb /usr/bin | |
| 39 | ``` | |
| 40 | ||
| 41 | ### GrapheneOS Files | |
| 42 | ||
| 43 | Next, we need the [GrapheneOS files](https://grapheneos.org/releases) for | |
| 44 | our device and model. For example, the Pixel 6 Pro is codenamed `raven` on the | |
| 45 | release page. | |
| 46 | ||
| 47 | Once we have the links, let's download them to our working directory: | |
| 48 | ||
| 49 | ```sh | |
| 50 | curl -O https://releases.grapheneos.org/factory.pub | |
| 51 | curl -0 https://releases.grapheneos.org/raven-factory-2022091400.zip | |
| 52 | curl -0 https://releases.grapheneos.org/raven-factory-2022091400.zip.sig | |
| 53 | ``` | |
| 54 | ||
| 55 | #### Validate Integrity | |
| 56 | ||
| 57 | In order to validate the integrity of the downloaded files, we will need the | |
| 58 | `signify` package and Graphene's `factory.pub` file. | |
| 59 | ||
| 60 | ```sh | |
| 61 | sudo dnf install signify | |
| 62 | ``` | |
| 63 | ||
| 64 | ```sh | |
| 65 | curl -O https://releases.grapheneos.org/factory.pub | |
| 66 | ``` | |
| 67 | ||
| 68 | Then we can validate the files and ensure that no data was corrupted or | |
| 69 | modified before it was saved to our device. | |
| 70 | ||
| 71 | ```sh | |
| 72 | signify -Cqp factory.pub -x raven-factory-2022091400.zip.sig && echo verified | |
| 73 | ``` | |
| 74 | ||
| 75 | #### Unzip Files | |
| 76 | ||
| 77 | Once the files are verified, we can unzip the Graphene image and enter the | |
| 78 | directory: | |
| 79 | ||
| 80 | ```sh | |
| 81 | unzip raven-factory-2022091400.zip && cd raven-factory-2022091400 | |
| 82 | ``` | |
| 83 | ||
| 84 | ## Installation Process | |
| 85 | ||
| 86 | ### Enable Developer Debugging & OEM Unlock | |
| 87 | ||
| 88 | Before we can actually flash anything to the phone, we will need to enable | |
| 89 | OEM Unlocking, as well as either USB Debugging or Wireless Debugging, depending on | |
| 90 | which method we will be using. | |
| 91 | ||
| 92 | To start, enable developer mode by going to `Settings` > `About` and tapping | |
| 93 | `Build Number` seven (7) times. You may need to enter your PIN to enable this | |
| 94 | mode. | |
| 95 | ||
| 96 | Once developer mode is enabled, go to `Settings` > `System` > `Devloper | |
| 97 | Options` and enable OEM Unlocking, as well as USB or Wireless Debugging. In my | |
| 98 | case, I chose USB Debugging and performed all actions via USB cable. | |
| 99 | ||
| 100 | Once these options are enabled, plug the phone into the computer and execute the | |
| 101 | following command: | |
| 102 | ||
| 103 | ```sh | |
| 104 | adb devices | |
| 105 | ``` | |
| 106 | ||
| 107 | If an unauthorized error occurs, make sure the USB mode on the phone is | |
| 108 | changed from charging to something like "File Transfer" or "PTP." | |
| 109 | You can find the USB mode in the notification tray. | |
| 110 | ||
| 111 | ### Reboot Device | |
| 112 | ||
| 113 | Once we have found the device via `adb`, we can either boot into the | |
| 114 | bootloader interface by holding the volume down button while the phone reboots | |
| 115 | or by executing the following command: | |
| 116 | ||
| 117 | ```sh | |
| 118 | adb reboot bootloader | |
| 119 | ``` | |
| 120 | ||
| 121 | ### Unlock the Bootloader | |
| 122 | ||
| 123 | The phone will reboot and load the bootloader screen upon startup. At this | |
| 124 | point, we are ready to start the actual flashing of GrapheneOS onto the device. | |
| 125 | ||
| 126 | **NOTE**: In my situation, I needed to use `sudo` with every `fastboot` | |
| 127 | command, but not with `adb` commands. I am not sure if this is standard or a | |
| 128 | Fedora quirk, but I'm documenting my commands verbatim in this post. | |
| 129 | ||
| 130 | First, we start by unlocking the bootloader so that we can load other ROMs: | |
| 131 | ||
| 132 | ```sh | |
| 133 | sudo fastboot flashing unlock | |
| 134 | ``` | |
| 135 | ||
| 136 | ### Flashing Factory Images | |
| 137 | ||
| 138 | Once the phone is unlocked, we can flash it with the `flash-all.sh` script found | |
| 139 | inside the `raven-factory-2022091400` folder we entered earlier: | |
| 140 | ||
| 141 | ```sh | |
| 142 | sudo ./flash-all.sh | |
| 143 | ``` | |
| 144 | ||
| 145 | This process should take a few minutes and will print informational messages as | |
| 146 | things progress. Avoid doing anything on the phone while this process is | |
| 147 | operating. | |
| 148 | ||
| 149 | ### Lock the Bootloader | |
| 150 | ||
| 151 | If everything was successful, the phone should reboot a few times and finally | |
| 152 | land back on the bootloader screen. At this point, we can re-lock the | |
| 153 | bootloader to enable full verified boot and protect the device from unwanted | |
| 154 | flashing or erasure of data. | |
| 155 | ||
| 156 | ```sh | |
| 157 | sudo fastboot flashing lock | |
| 158 | ``` | |
| 159 | ||
| 160 | Once done, the device will be wiped and ready for a fresh set-up! | |
blog/2022-10-04-mtp-linux.org added +79
| @@ -0,0 +1,79 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-10-04 | |
| 3 | title = "How to Mount an MTP Mobile Device on Linux" | |
| 4 | description = "Learn how to easily mount Android MTP smartphones on Linux using the command line." | |
| 5 | +++ | |
| 6 | ||
| 7 | I recently ran into trouble attempting to mount my GrapheneOS phone to my laptop | |
| 8 | running Fedora Linux via the [Media Transfer | |
| 9 | Protocol](https://en.wikipedia.org/wiki/Media_transfer_protocol) (MTP) and | |
| 10 | discovered a simple and effective solution. | |
| 11 | ||
| 12 | ## Use a USB 3.0 Port | |
| 13 | ||
| 14 | First, ensure that the device was plugged in to the laptop through a | |
| 15 | USB 3.0 port, if possible. From a brief glance online, it seems that USB 2.0 | |
| 16 | ports may cause issues with dropped connections over MTP. This is purely | |
| 17 | anecdotal since I don't have any evidence to link showing that USB 2.0 causes | |
| 18 | issues, but I can confirm that switching to a USB 3.0 port seemed to cut out | |
| 19 | most | |
| 20 | of my issues. | |
| 21 | ||
| 22 | ## Switch USB Preferences to MTP | |
| 23 | ||
| 24 | Secondly, you need to ensure that the phone's USB preferences/mode is | |
| 25 | changed to MTP or File Transfer once the phone is plugged in. Other modes will | |
| 26 | not allow you to access the phone's file system. | |
| 27 | ||
| 28 | ## Install `jmtpfs` | |
| 29 | ||
| 30 | Next, I used the `jmtpfs` package to mount my phone to my laptop. There are | |
| 31 | other packages that exist, but this one worked perfectly for me. On Fedora | |
| 32 | Linux, you can install it like this: | |
| 33 | ||
| 34 | ```sh | |
| 35 | sudo dnf install jmtpfs -y | |
| 36 | ``` | |
| 37 | ||
| 38 | ## Create a Mount Point | |
| 39 | ||
| 40 | Once you have the package installed, you just need to create a folder for the | |
| 41 | device to use as a mount point. In my case, I used `/mnt/pixel`: | |
| 42 | ||
| 43 | ```sh | |
| 44 | sudo mkdir /mnt/pixel | |
| 45 | sudo chown -R $USER:$USER /mnt/pixel | |
| 46 | ``` | |
| 47 | ||
| 48 | ## Mount & Access the Phone's File System | |
| 49 | ||
| 50 | Finally, plug-in and mount the device, and you should be able to see all | |
| 51 | storage (internal and external) inside your new folder! | |
| 52 | ||
| 53 | ```sh | |
| 54 | jmtpfs /mnt/pixel | |
| 55 | ``` | |
| 56 | ||
| 57 | The output should look something like this: | |
| 58 | ||
| 59 | ```sh | |
| 60 | Device 0 (VID=18d1 and PID=4ee1) is a Google Inc Nexus/Pixel (MTP). | |
| 61 | Android device detected, assigning default bug flags | |
| 62 | ``` | |
| 63 | ||
| 64 | Now you are mounted and can do anything you'd like with the device's files: | |
| 65 | ||
| 66 | ```sh | |
| 67 | cd /mnt/pixel | |
| 68 | ls -lha | |
| 69 | ``` | |
| 70 | ||
| 71 | From here, you will be able to see any internal or external storage available on | |
| 72 | the device: | |
| 73 | ||
| 74 | ```sh | |
| 75 | total 0 | |
| 76 | drwxr-xr-x. 3 user user 0 Jan 1 1970 . | |
| 77 | drwxr-xr-x. 1 root root 10 Oct 4 13:29 .. | |
| 78 | drwxr-xr-x. 16 user user 0 Apr 21 4426383 'Internal shared storage' | |
| 79 | ``` | |
blog/2022-10-20-syncthing.org added +166
| @@ -0,0 +1,166 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-10-20 | |
| 3 | title = "Syncthing: A Minimal Self-Hosted Cloud Storage Solution" | |
| 4 | description = "In this post, I walk through the Syncthing software and how to set it up on multiple devices." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## An Overview of Syncthing | |
| 8 | ||
| 9 | If you've been looking around the self-hosted cloud storage space for a while, | |
| 10 | you've undoubtedly run into someone suggesting | |
| 11 | [Syncthing](https://syncthing.net) as an option. However, it is an unusual | |
| 12 | alternative for those users out there who are used to having a centralized cloud | |
| 13 | server that serves as the "controller" of the data and interacts with clients on | |
| 14 | devices to fetch files. | |
| 15 | ||
| 16 | This post is a walkthrough of the Syncthing software, how I set up my personal | |
| 17 | storage, and some pros and cons of using the software. | |
| 18 | ||
| 19 | ## Installing Syncthing | |
| 20 | ||
| 21 | To install Syncthing, visit the [Downloads](https://syncthing.net/downloads/) | |
| 22 | page or install via your device's package manager. | |
| 23 | ||
| 24 | ### Server & Desktop | |
| 25 | ||
| 26 | You can install Syncthing on servers and desktops via the Downloads page linked | |
| 27 | above or via the command-line. | |
| 28 | ||
| 29 | For Debian-based distros: | |
| 30 | ||
| 31 | ```sh | |
| 32 | sudo apt install syncthing | |
| 33 | ``` | |
| 34 | ||
| 35 | For Fedora-based distros: | |
| 36 | ||
| 37 | ```sh | |
| 38 | sudo dnf install syncthing | |
| 39 | ``` | |
| 40 | ||
| 41 | ### Mobile | |
| 42 | ||
| 43 | Syncthing for Android is available on | |
| 44 | [F-Droid](https://f-droid.org/packages/com.nutomic.syncthingandroid/) and | |
| 45 | [Google Play](https://play.google.com/store/apps/details?id=com.nutomic.syncthingandroid). | |
| 46 | Syncthing does not have an official iOS client, but there is a third-party | |
| 47 | client called [Möbius | |
| 48 | Sync](https://apps.apple.com/us/app/m%C3%B6bius-sync/id1539203216). | |
| 49 | ||
| 50 | ## How Does Syncthing Work? | |
| 51 | ||
| 52 | To start, I wanted to include the main marketing blurb from their website: | |
| 53 | ||
| 54 | > Syncthing is a continuous file synchronization program. It synchronizes files | |
| 55 | > between two or more computers in real time, safely protected from prying eyes. | |
| 56 | > Your data is your data alone, and you deserve to choose where it is stored, | |
| 57 | > whether it is shared with some third party, and how it's transmitted over the | |
| 58 | > internet. | |
| 59 | ||
| 60 | Let's break this apart and add in some other details to help explain what | |
| 61 | exactly Syncthing does in order to sync files between devices. | |
| 62 | ||
| 63 | ### Local Syncthing Server(s) | |
| 64 | ||
| 65 | Syncthing syncs files between multiple devices by creating a local server on | |
| 66 | each device. These local servers handle a few different things, such as | |
| 67 | watching files and directories for changes, hosting an administrative GUI | |
| 68 | website, and authenticating with connected devices. | |
| 69 | ||
| 70 | You can also start, stop, and restart the Syncthing server via the command-line | |
| 71 | or web dashboard. If you're running Syncthing on a device with `systemd`, you | |
| 72 | can use the following commands: | |
| 73 | ||
| 74 | ```sh | |
| 75 | sudo systemctl start syncthing@username.service | |
| 76 | sudo systemctl restart syncthing@username.service | |
| 77 | sudo systemctl stop syncthing@username.service | |
| 78 | ``` | |
| 79 | ||
| 80 | ### Syncthing Dashboard | |
| 81 | ||
| 82 | This biggest part of Syncthing is the admin GUI website that runs on each | |
| 83 | device (note that mobile devices will use the Syncthing app rather than the | |
| 84 | web GUI). The admin GUI is available through the web browser on the | |
| 85 | local device that is running Syncthing - simply go to `http://localhost:8384` | |
| 86 | or `http://127.0.0.1:8384`. This web page is the place where you will change | |
| 87 | settings, add/modify synced files, and add/modify connected devices. | |
| 88 | ||
| 89 | Here's an example web GUI dashboard: | |
| 90 | ||
| 91 |  | |
| 92 | ||
| 93 | ### Remote Devices | |
| 94 | ||
| 95 | A cloud storage solution wouldn't be very useful if you aren't able to share | |
| 96 | data among various devices. Syncthing does this by sharing Device IDs to connect | |
| 97 | servers, and then by manually sharing Folders with devices that have been | |
| 98 | connected. | |
| 99 | ||
| 100 | For instance, if you have a laptop running Syncthing and then install the | |
| 101 | Syncthing mobile app on a phone, you could scan the laptop's QR code for Device | |
| 102 | ID and then accept the authentication on the laptop's dashboard. Next, you can | |
| 103 | use either device to select a folder for sharing and dictating which device | |
| 104 | should send, receive, or both. | |
| 105 | ||
| 106 | When you connect devices, you can set one device as an "Introducer," which can | |
| 107 | add devices from the introducer to the device list, for mutually shared folders. | |
| 108 | You can also configure Auto Accept, compression, rate limits, and more settings | |
| 109 | per device. | |
| 110 | ||
| 111 | ## My Personal Cloud Storage Set-up | |
| 112 | ||
| 113 | Personally, I use a model similar to a traditional cloud storage service. I have | |
| 114 | a "centralized" server running 24/7 that acts as an Introducer for my Syncthing | |
| 115 | network. I think of this as my main storage and all other devices as tertiary | |
| 116 | client devices. I will likely add additional servers as backups as time goes on | |
| 117 | so that I don't have to rely on my laptop or phone as the only backups. | |
| 118 | ||
| 119 | Currently, I have one desktop and one mobile device connected to the network, | |
| 120 | both running intermittently as they are not powered-on 24/7. | |
| 121 | ||
| 122 | The initial set-up of the software was easy enough, but data transfer rates were | |
| 123 | incredibly slow for me due to the Wi-Fi. Instead, I plugged my laptop into the | |
| 124 | ethernet network that my server is on and manually copied my folders over to the | |
| 125 | server with `scp`. Once complete, Syncthing validated that all files were there | |
| 126 | and not missing, and it did not need to transfer any data through the WAN. | |
| 127 | ||
| 128 | As slow as the transfer was going, this probably saved me a few days of waiting | |
| 129 | for my ~100GB sync. | |
| 130 | ||
| 131 | ## Pros & Cons | |
| 132 | ||
| 133 | I've put together a short list of pros and cons for Syncthing. I thought about | |
| 134 | my experiences with Nextcloud, WebDAV, proprietary services (Google Drive, | |
| 135 | iCloud, etc.), and privacy-focused cloud solutions (pCloud, Tresorit, etc.). | |
| 136 | ||
| 137 | **Pros:** | |
| 138 | ||
| 139 | - I've faced no data loss at all through my two-month trial run. | |
| 140 | - No third-parties store your data on their servers. | |
| 141 | - You have full control over your data and can take your data and leave at any | |
| 142 | time. | |
| 143 | - It's possible to encrypt client-side easily with software like Cryptomator. | |
| 144 | - No proprietary clients or mounted volumes, just plain files and folders. | |
| 145 | ||
| 146 | **Cons:** | |
| 147 | ||
| 148 | - The learning curve is steeper than traditional cloud services and is | |
| 149 | focused on a technical audience. | |
| 150 | - If a device needs to modify files in a Folder, the devices will need to | |
| 151 | sync ALL files from the folder, which may be large. To avoid size | |
| 152 | restraints, split large folders into smaller folders for syncing. | |
| 153 | - Syncing can be slow due to the clients/servers initially connecting or | |
| 154 | re-connecting after sleeping. | |
| 155 | - Multiple personal devices are required and require the user to own or rent | |
| 156 | them as no third-party servers are involved in the storage of data. | |
| 157 | ||
| 158 | Overall, I've had a great experience with Syncthing so far. I've had no data | |
| 159 | loss, syncing has been quick and easy when changes are made to files, device | |
| 160 | connections are reliable, and I love the freedom of controlling the clients and | |
| 161 | servers as I choose. | |
| 162 | ||
| 163 | Not to mention that I appreciate that I - or someone else - could pull the | |
| 164 | Syncthing [source code](https://github.com/syncthing) and continue | |
| 165 | development/support if the Syncthing Foundation decides to stop developing the | |
| 166 | software or sells the business. | |
blog/2022-10-22-alpine-linux.org added +264
| @@ -0,0 +1,264 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-10-22 | |
| 3 | title = "Alpine Linux: My New Server OS" | |
| 4 | description = "In this post, I explain Alpine Linux's benefits as a server OS and explain my current setup." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Alpine Linux | |
| 8 | ||
| 9 | [Alpine Linux](https://alpinelinux.org) is a very small distro, built on musl | |
| 10 | libc and busybox. It uses ash as the default shell, OpenRC as the init | |
| 11 | system, and apk as the package manager. According to their website, an Alpine | |
| 12 | container "requires no more than 8 MB and a minimal installation to disk | |
| 13 | requires around 130 MB of storage." An actual bare metal machine is recommended | |
| 14 | to have 100 MB of RAM and 0-700 MB of storage space. | |
| 15 | ||
| 16 | Historically, I've used Ubuntu's minimal installation image as my server OS for | |
| 17 | the last five years. Ubuntu worked well and helped as my original server | |
| 18 | contained an nVidia GPU and no onboard graphics, so quite a few distros won't | |
| 19 | boot or install without a lot of tinkering. | |
| 20 | ||
| 21 | Alpine has given me a huge increase in performance across my Docker apps and | |
| 22 | Nginx websites. CPU load for the new server I'm using to test Alpine hovers | |
| 23 | around 0-5% on average with an Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz. | |
| 24 | ||
| 25 | The only services I haven't moved over to Alpine are Plex Media Server and | |
| 26 | Syncthing, which may increase CPU load quite a bit depending on how many | |
| 27 | streams are running. | |
| 28 | ||
| 29 | ### Installation | |
| 30 | ||
| 31 | In terms of installation, Alpine has an incredibly useful | |
| 32 | [wiki](https://wiki.alpinelinux.org/wiki/Installation) that will guide a user | |
| 33 | throughout the installation and post-installation processes, as well as various | |
| 34 | other articles and guides. | |
| 35 | ||
| 36 | To install Alpine, find an appropriate [image to | |
| 37 | download](https://alpinelinux.org/downloads/) and flash it to a USB using | |
| 38 | software such as Rufus or Etcher. I opted to use the Standard image for my | |
| 39 | x86_64 architecture. | |
| 40 | ||
| 41 | Once the USB is ready, plug it into the machine and reboot. | |
| 42 | Note that you may have to use a key such as `Esc` or `F1-12` to access the | |
| 43 | boot menu. | |
| 44 | The Alpine Linux terminal will load quickly and for a login. | |
| 45 | ||
| 46 | To log in to the installation image, use the `root` account; there is no | |
| 47 | password. Once logged-in, execute the setup command: | |
| 48 | ||
| 49 | ```sh | |
| 50 | setup-alpine | |
| 51 | ``` | |
| 52 | ||
| 53 | The setup script will ask a series of questions to configure the system. Be sure | |
| 54 | to answer carefully or else you may have to re-configure the system after boot. | |
| 55 | ||
| 56 | - Keyboard Layout (Local keyboard language and usage mode, e.g., us and variant | |
| 57 | of us-nodeadkeys.) | |
| 58 | - Hostname (The name for the computer.) | |
| 59 | - Network (For example, automatic IP address discovery with the "DHCP" | |
| 60 | protocol.) | |
| 61 | - DNS Servers (Domain Name Servers to query. For privacy reasons, it is NOT | |
| 62 | recommended to route every local request to servers like Google's 8.8.8.8 .) | |
| 63 | - Timezone | |
| 64 | - Proxy (Proxy server to use for accessing the web. Use "none" for direct | |
| 65 | connections to the internet.) | |
| 66 | - Mirror (From where to download packages. Choose the organization you trust | |
| 67 | giving your usage patterns to.) | |
| 68 | - SSH (Secure SHell remote access server. "Openssh" is part of the default | |
| 69 | install image. Use "none" to disable remote login, e.g. on laptops.) | |
| 70 | - NTP (Network Time Protocol client used for keeping the system clock in sync | |
| 71 | with a time-server. Package "chrony" is part of the default install image.) | |
| 72 | - Disk Mode (Select between diskless (disk="none"), "data" or "sys", as | |
| 73 | described above.) | |
| 74 | ||
| 75 | Once the setup script is finished, be sure to reboot the machine and remove the | |
| 76 | USB device. | |
| 77 | ||
| 78 | ```sh | |
| 79 | reboot | |
| 80 | ``` | |
| 81 | ||
| 82 | ### Post-Installation | |
| 83 | ||
| 84 | There are many things you can do once your Alpine Linux system is up and | |
| 85 | running, and it largely depends on what you'll use the machine for. | |
| 86 | I'm going to walk through my personal post-installation setup for my web server. | |
| 87 | ||
| 88 | #### Upgrade the System | |
| 89 | ||
| 90 | First, login as `root` in order to update and upgrade the system: | |
| 91 | ||
| 92 | ```sh | |
| 93 | apk -U upgrade | |
| 94 | ``` | |
| 95 | ||
| 96 | #### Adding a User | |
| 97 | ||
| 98 | I needed to add a user so that I don't need to log in as root. | |
| 99 | Note that if you're used to using the `sudo` command, you will now need to | |
| 100 | use the `doas` command on Alpine Linux. | |
| 101 | ||
| 102 | ```sh | |
| 103 | apk add doas | |
| 104 | adduser <username> | |
| 105 | adduser <username> wheel | |
| 106 | ``` | |
| 107 | ||
| 108 | You can now log out and log back in using the newly-created user: | |
| 109 | ||
| 110 | ```sh | |
| 111 | exit | |
| 112 | ``` | |
| 113 | ||
| 114 | #### Enable Community Packages | |
| 115 | ||
| 116 | In order to install more common packages that aren't found in the `main` | |
| 117 | repository, you will need to enable the `community` repository: | |
| 118 | ||
| 119 | ```sh | |
| 120 | doas nano /etc/apk/repositories | |
| 121 | ``` | |
| 122 | ||
| 123 | Uncomment the community line for whichever version of Alpine you're running: | |
| 124 | ||
| 125 | ```sh | |
| 126 | /media/usb/apks | |
| 127 | http://dl-cdn.alpinelinux.org/alpine/v3.16/main | |
| 128 | http://dl-cdn.alpinelinux.org/alpine/v3.16/community | |
| 129 | #http://dl-cdn.alpinelinux.org/alpine/edge/main | |
| 130 | #http://dl-cdn.alpinelinux.org/alpine/edge/community | |
| 131 | #http://dl-cdn.alpinelinux.org/alpine/edge/testing | |
| 132 | ``` | |
| 133 | ||
| 134 | #### Install Required Packages | |
| 135 | ||
| 136 | Now that the community packages are available, you can install any packages you | |
| 137 | need. In my case, I installed the web server packages I need for my services: | |
| 138 | ||
| 139 | ```sh | |
| 140 | doas apk add nano nginx docker docker-compose ufw | |
| 141 | ``` | |
| 142 | ||
| 143 | #### SSH | |
| 144 | ||
| 145 | If you didn't install OpenSSH as part of the installation, you can do so now: | |
| 146 | ||
| 147 | ```sh | |
| 148 | doas apk add openssh | |
| 149 | ``` | |
| 150 | ||
| 151 | Next, either create a new key or copy your SSH key to the server from your | |
| 152 | current machines: | |
| 153 | ||
| 154 | ```sh | |
| 155 | # Create a new key | |
| 156 | ssh-keygen | |
| 157 | ``` | |
| 158 | ||
| 159 | If you need to copy an existing SSH key from a current machine: | |
| 160 | ||
| 161 | ```sh | |
| 162 | # Copy key from existing machines | |
| 163 | ssh-copy-id <username>@<ip_address> | |
| 164 | ``` | |
| 165 | ||
| 166 | #### Firewall | |
| 167 | ||
| 168 | Lastly, I installed `ufw` above as my firewall. | |
| 169 | To set up, default to deny incoming and allow outgoing connections. | |
| 170 | Then selectively allow other ports or apps as needed. | |
| 171 | ||
| 172 | ```sh | |
| 173 | doas ufw default deny incoming | |
| 174 | doas ufw default allow outgoing | |
| 175 | doas ufw allow SSH | |
| 176 | doas ufw allow "WWW Full" | |
| 177 | doas ufw allow 9418 # Git server port | |
| 178 | ``` | |
| 179 | ||
| 180 | #### Change Hostname | |
| 181 | ||
| 182 | If you don't like the hostname set during installation, you just need to edit | |
| 183 | two files. First, edit the simple hostname file: | |
| 184 | ||
| 185 | ```sh | |
| 186 | doas nano /etc/hostname | |
| 187 | ``` | |
| 188 | ||
| 189 | ```sh | |
| 190 | <hostname> | |
| 191 | ``` | |
| 192 | ||
| 193 | Next, edit the `hosts` file: | |
| 194 | ||
| 195 | ```sh | |
| 196 | doas nano /etc/hosts | |
| 197 | ``` | |
| 198 | ||
| 199 | ```sh | |
| 200 | 127.0.0.1 <hostname>.local <hostname> localhost.local localhost | |
| 201 | ::1 <hostname> <hostname>.local | |
| 202 | ``` | |
| 203 | ||
| 204 | ## Nginx Web Server | |
| 205 | ||
| 206 | To set up my web server, I simply created the `www` user and created the | |
| 207 | necessary files. | |
| 208 | ||
| 209 | ```sh | |
| 210 | doas adduser -D -g 'www' www | |
| 211 | mkdir /www | |
| 212 | doas mkdir /www | |
| 213 | doas chown -R www:www /var/lib/nginx/ | |
| 214 | doas chown -R www:www /www | |
| 215 | ``` | |
| 216 | ||
| 217 | If you're running a simple webroot, you can alter the main `nginx.conf` file. | |
| 218 | Otherwise, you can drop configuration files in the following directory. You | |
| 219 | don't need to enable or symlink the configuration file like you do in other | |
| 220 | systems. | |
| 221 | ||
| 222 | ```sh | |
| 223 | doas nano /etc/nginx/http.d/example_website.conf | |
| 224 | ``` | |
| 225 | ||
| 226 | Once the configuration is set and pointed at the `/www` directory to serve | |
| 227 | files, enable the Nginx service: | |
| 228 | ||
| 229 | ```sh | |
| 230 | # Note that 'default' must be included or Nginx will not start on boot | |
| 231 | doas rc-update add nginx default | |
| 232 | ``` | |
| 233 | ||
| 234 | ## Docker Containers | |
| 235 | ||
| 236 | Docker works exactly the same as other systems. Either execute a `docker run` | |
| 237 | command or create a `docker-compose.yml` file and do `docker-compose up -d`. | |
| 238 | ||
| 239 | ## Git Server | |
| 240 | ||
| 241 | I went in-depth on how to self-host a git server in another post: | |
| 242 | [Self-Hosting a Personal Git Server](/blog/git-server/). | |
| 243 | ||
| 244 | However, there are a few differences with Alpine. First note that in order to | |
| 245 | change the `git` user's shell, you must do a few things a little different: | |
| 246 | ||
| 247 | ```sh | |
| 248 | doas apk add libuser | |
| 249 | doas touch /etc/login.defs | |
| 250 | doas mkdir /etc/default | |
| 251 | doas touch /etc/default/useradd | |
| 252 | doas lchsh git | |
| 253 | ``` | |
| 254 | ||
| 255 | ## Thoughts on Alpine | |
| 256 | ||
| 257 | So far, I love Alpine Linux. I have no complaints about anything at this point, | |
| 258 | but I'm not completely finished with the migration yet. Once I'm able to upgrade | |
| 259 | my hardware to a rack-mounted server, I will migrate Plex and Syncthing over to | |
| 260 | Alpine as well - possibly putting Plex into a container or VM. | |
| 261 | ||
| 262 | The performance is stellar, the `apk` package manager is seamless, and system | |
| 263 | administration tasks are effortless. My only regret is that I didn't | |
| 264 | install Alpine sooner. | |
blog/2022-10-30-linux-display-manager.org added +78
| @@ -0,0 +1,78 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-10-30 | |
| 3 | title = "How to Disable or Change the Display Manager on Void Linux" | |
| 4 | description = "Learn how to change the default dm on Void Linux or disable it completely." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Display Manager Services | |
| 8 | ||
| 9 | In order to change the [display | |
| 10 | manager](https://en.wikipedia.org/wiki/Display_manager) on Void Linux - or any | |
| 11 | other Linux distro - you need to identify the currently enabled display manager. | |
| 12 | ||
| 13 | ### Disabling the Current Display Manager | |
| 14 | ||
| 15 | Void Linux only has one ISO available for download with a pre-built display | |
| 16 | manager at the time of this post: the XFCE ISO. If you've installed this | |
| 17 | version, the pre-assigned display manager is `lxdm`. If you installed another | |
| 18 | display manager, replace `lxdm` in the following command with the display | |
| 19 | manager you have installed. | |
| 20 | ||
| 21 | To disable `lxdm`, simply remove the service symlink: | |
| 22 | ||
| 23 | ```sh | |
| 24 | sudo rm /var/service/lxdm | |
| 25 | ``` | |
| 26 | ||
| 27 | ### Enabling a New Display Manager | |
| 28 | ||
| 29 | If you want to enable a new display manager, you can do so after `lxdm` is | |
| 30 | disabled. Make sure to replace `<new_display_manager>` with your new DM, such as | |
| 31 | `gdm`, `xdm`, etc. | |
| 32 | ||
| 33 | ||
| 34 | ```sh | |
| 35 | sudo ln -s /etc/sv/<new_display_manager> /var/service | |
| 36 | ``` | |
| 37 | ||
| 38 | ## Set Up `.xinitrc` | |
| 39 | ||
| 40 | Depending on your setup, you may need to create a few X files, such as | |
| 41 | `~/.xinitrc`. For my personal set-up, I created this file to launch the i3wm as | |
| 42 | my desktop. | |
| 43 | ||
| 44 | ```sh | |
| 45 | nano ~/.xinitrc | |
| 46 | ``` | |
| 47 | ||
| 48 | ```sh | |
| 49 | #!/bin/sh | |
| 50 | ||
| 51 | exec i3 | |
| 52 | ``` | |
| 53 | ||
| 54 | If you run a desktop other than i3, simply replace `i3` with the shell command | |
| 55 | that launches that desktop. | |
| 56 | ||
| 57 | ## Set Up Your Shell Profile | |
| 58 | ||
| 59 | Finally, in order to automatically launch an X session upon login, you will need | |
| 60 | to edit the `.bash_profile` (bash) or `.zprofile` (zsh) files for your | |
| 61 | shell: | |
| 62 | ||
| 63 | ```sh | |
| 64 | nano ~/.zprofile | |
| 65 | ``` | |
| 66 | ||
| 67 | Add the following snippet to the end of the shell profile file. This will | |
| 68 | execute the `startx` command upon login. | |
| 69 | ||
| 70 | ```sh | |
| 71 | if [ -z "${DISPLAY}" ] && [ "${XDG_VTNR}" -eq 1 ]; then | |
| 72 | exec startx | |
| 73 | fi | |
| 74 | ``` | |
| 75 | ||
| 76 | Alternatively, you can ignore this step and simply choose to manually execute | |
| 77 | `startx` upon login. This can be useful if you have issues with your desktop or | |
| 78 | like to manually launch different desktops by choice. | |
blog/2022-11-07-matrix-synapse.org added +212
| @@ -0,0 +1,212 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-11-07 | |
| 3 | title = "Self-Hosting Matrix Synapse on Alpine Linux" | |
| 4 | description = "A short guide on how I was able to get Synapse working on Alpine Linux." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Synpase | |
| 8 | ||
| 9 | If you're reading this, you likely know that | |
| 10 | [Synapse](https://github.com/matrix-org/synapse/) is a popular | |
| 11 | [Matrix](https://matrix.org/) home server software that allows users to run | |
| 12 | their own Matrix home server. | |
| 13 | ||
| 14 | This post is a short guide describing how I was able to get Synapse working in a | |
| 15 | minimally-usable state on Alpine Linux. | |
| 16 | ||
| 17 | ## Installation Process | |
| 18 | ||
| 19 | ### Dependencies | |
| 20 | ||
| 21 | First, since there is no Alpine-specific package for Synapse, we need to ensure | |
| 22 | that Alpine has the required dependencies for the Python-based installation | |
| 23 | method. | |
| 24 | ||
| 25 | ```sh | |
| 26 | doas apk -U update | |
| 27 | doas apk add python3 py3-virtualenv | |
| 28 | ``` | |
| 29 | ||
| 30 | Next, we need to set up a Python virtual environment for Synapse: | |
| 31 | ||
| 32 | ```sh | |
| 33 | mkdir -p ~/synapse && cd ~/synapse | |
| 34 | virtualenv -p python3 ~/synapse/env | |
| 35 | source ~/synapse/env/bin/activate | |
| 36 | pip install --upgrade pip | |
| 37 | pip install --upgrade setuptools | |
| 38 | pip install matrix-synapse | |
| 39 | ``` | |
| 40 | ||
| 41 | ### Running Synapse | |
| 42 | ||
| 43 | Once installed, running Synapse is easy. Simply execute the following command, | |
| 44 | replacing `example.com` with the domain name that will be used with this | |
| 45 | home server. This will generate the configuration files needed to run the | |
| 46 | server. | |
| 47 | ||
| 48 | ```sh | |
| 49 | python -m synapse.app.homeserver \ | |
| 50 | --server-name example.com \ | |
| 51 | --config-path homeserver.yaml \ | |
| 52 | --generate-config \ | |
| 53 | --report-stats=no | |
| 54 | ``` | |
| 55 | ||
| 56 | Once the configuration is generated, we can start up the Synapse server: | |
| 57 | ||
| 58 | ```sh | |
| 59 | synctl start | |
| 60 | ``` | |
| 61 | ||
| 62 | ### Configuring Synapse | |
| 63 | ||
| 64 | To make any change to Synapse, we need to edit the `YAML` configuration file: | |
| 65 | ||
| 66 | ```sh | |
| 67 | nano ~/synapse/homeserver.yaml | |
| 68 | ``` | |
| 69 | ||
| 70 | For now, we just need to ensure the `server_name` is accurate. However, there | |
| 71 | are a lot of other configuration options found in the [Configuring | |
| 72 | Synapse](https://matrix-org.github.io/synapse/develop/usage/configuration/config_documentation.html) | |
| 73 | documentation that can be enabled/disabled at any point. | |
| 74 | ||
| 75 | ```yaml | |
| 76 | server_name: "example.com" | |
| 77 | ``` | |
| 78 | ||
| 79 | Make sure to restart Synapse when you make changes to the configuration: | |
| 80 | ||
| 81 | ```sh | |
| 82 | synctl restart | |
| 83 | ``` | |
| 84 | ||
| 85 | ### Nginx Reverse-Proxy | |
| 86 | ||
| 87 | To ensure that Synapse is reachable from the public, we need to connect our | |
| 88 | domain to the Synapse server. | |
| 89 | In my case, I use a Nginx reverse-proxy for this purpose. | |
| 90 | ||
| 91 | To use Nginx, we need to create a reverse-proxy configuration file: | |
| 92 | ||
| 93 | ```sh | |
| 94 | doas nano /etc/nginx/http.d/example.com.conf | |
| 95 | ``` | |
| 96 | ||
| 97 | If you already have TLS certificates for this domain (`example.com`), you can | |
| 98 | simply use the SSL configuration and point toward your TLS certificates. | |
| 99 | ||
| 100 | ```conf | |
| 101 | server { | |
| 102 | listen 443 ssl http2; | |
| 103 | listen [::]:443 ssl http2; | |
| 104 | ||
| 105 | # For the federation port | |
| 106 | listen 8448 ssl http2; | |
| 107 | listen [::]:8448 ssl http2; | |
| 108 | ||
| 109 | server_name example.com; | |
| 110 | ||
| 111 | location ~ ^(/_matrix|/_synapse/client) { | |
| 112 | # note: do not add a path (even a single /) after the port in `proxy_pass`, | |
| 113 | # otherwise nginx will canonicalise the URI and cause signature verification | |
| 114 | # errors. | |
| 115 | proxy_pass http://localhost:8008; | |
| 116 | proxy_set_header X-Forwarded-For $remote_addr; | |
| 117 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 118 | proxy_set_header Host $host; | |
| 119 | ||
| 120 | # Nginx by default only allows file uploads up to 1M in size | |
| 121 | # Increase client_max_body_size to match max_upload_size defined in homeserver.yaml | |
| 122 | client_max_body_size 50M; | |
| 123 | } | |
| 124 | ||
| 125 | ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; | |
| 126 | ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; | |
| 127 | access_log /var/log/nginx/matrix.access.log; | |
| 128 | } | |
| 129 | ||
| 130 | server { | |
| 131 | if ($host = example.com) { | |
| 132 | return 301 https://$host$request_uri; | |
| 133 | } | |
| 134 | ||
| 135 | server_name example.com; | |
| 136 | listen 80; | |
| 137 | return 404; | |
| 138 | } | |
| 139 | ``` | |
| 140 | ||
| 141 | If you need to generate TLS certificates (I recommend | |
| 142 | [Certbot](https://certbot.eff.org/)), you'll need a more minimal Nginx conf file | |
| 143 | before you can use the TLS-enabled example above. Instead, use this | |
| 144 | configuration file during the Certbot certificate generation process: | |
| 145 | ||
| 146 | ```conf | |
| 147 | server { | |
| 148 | server_name example.com; | |
| 149 | location / { | |
| 150 | try_files $uri $uri/ =404; | |
| 151 | } | |
| 152 | listen 80; | |
| 153 | } | |
| 154 | ``` | |
| 155 | ||
| 156 | Once you're done editing the Nginx conf file, restart Nginx: | |
| 157 | ||
| 158 | ```sh | |
| 159 | doas rc-service nginx restart | |
| 160 | ``` | |
| 161 | ||
| 162 | If you still need to generate TLS certificates, run `certbot` now and obtain the | |
| 163 | certificates. Certbot will ask if you want to use a webroot or spin up a | |
| 164 | temporary web server. I **highly** recommend using the temporary web server due | |
| 165 | to the many issues with using a webroot. | |
| 166 | ||
| 167 | You will need to stop Nginx in order to user the temporary web server option | |
| 168 | with Certbot: | |
| 169 | ||
| 170 | ```sh | |
| 171 | # Stop Nginx so certbot can spin up a temp webserver for cert generation | |
| 172 | doas rc-service nginx stop | |
| 173 | doas certbot certonly -v | |
| 174 | doas rc-service nginx start | |
| 175 | ``` | |
| 176 | ||
| 177 | ### Open Firewall & Router Ports | |
| 178 | ||
| 179 | If you use a firewall on the server, open the `8448` port for discovery and | |
| 180 | federation, as well as the normal web server ports if you're using a reverse | |
| 181 | proxy. If you want additional services, such as voice calls, you will need | |
| 182 | to read the Synapse documentation to see which ports need to be opened for those | |
| 183 | features. | |
| 184 | ||
| 185 | Here's an example of the Universal Firewall (UFW) software: | |
| 186 | ||
| 187 | ```sh | |
| 188 | # Matrix port | |
| 189 | doas ufw allow 8448 | |
| 190 | # Standard web server ports | |
| 191 | doas ufw allow "Nginx Full" | |
| 192 | ``` | |
| 193 | ||
| 194 | Remember to forward any Synapse ports, such as `8448`, `80`, and `443`, in your | |
| 195 | Router from the internet to your server's IP address. | |
| 196 | ||
| 197 | ### Adding Matrix Users | |
| 198 | ||
| 199 | Finally, if you didn't enable public registration in the `homeserver.yaml` file, | |
| 200 | you can manually create users via the command-line: | |
| 201 | ||
| 202 | ```sh | |
| 203 | cd ~/synapse | |
| 204 | register_new_matrix_user -c homeserver.yaml | |
| 205 | ``` | |
| 206 | ||
| 207 | Remember that the format for federated Matrix usernames is | |
| 208 | `@username:example.com` when logging in to client applications. | |
| 209 | ||
| 210 | Once Synapse is running, and you have a username, you are ready to log in to a | |
| 211 | Matrix client and start sending messages, joining rooms, and utilizing your very | |
| 212 | own Matrix server. | |
blog/2022-11-11-nginx-tmp-errors.org added +76
| @@ -0,0 +1,76 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-11-11 | |
| 3 | title = "Fixing Permission Errors in /var/lib/nginx" | |
| 4 | description = "A quick fix for Nginx permission errors on the tmp directory." | |
| 5 | +++ | |
| 6 | ||
| 7 | *This is a brief post so that I personally remember the solution as it has | |
| 8 | occurred multiple times for me.* | |
| 9 | ||
| 10 | ## The Problem | |
| 11 | ||
| 12 | After migrating to a new server OS, I started receiving quite a few permission | |
| 13 | errors like the one below. These popped up for various different websites I'm | |
| 14 | serving via Nginx on this server, but did not prevent the website from loading. | |
| 15 | ||
| 16 | I found the errors in the standard log file: | |
| 17 | ||
| 18 | ```sh | |
| 19 | cat /var/log/nginx/error.log | |
| 20 | ``` | |
| 21 | ||
| 22 | ```sh | |
| 23 | 2022/11/11 11:30:34 [crit] 8970#8970: *10 open() "/var/lib/nginx/tmp/proxy/3/00/0000000003" failed (13: Permission denied) while reading upstream, client: 169.150.203.10, server: cyberchef.example.com, request: "GET /assets/main.css HTTP/2.0", upstream: "http://127.0.0.1:8111/assets/main.css", host: "cyberchef.example.com", referrer: "https://cyberchef.example.com/" | |
| 24 | ``` | |
| 25 | ||
| 26 | You can see that the error is `13: Permission denied` and it occurs in the | |
| 27 | `/var/lib/nginx/tmp/` directory. In my case, I had thousands of errors where | |
| 28 | Nginx was denied permission to read/write files in this directory. | |
| 29 | ||
| 30 | So how do I fix it? | |
| 31 | ||
| 32 | ## The Solution | |
| 33 | ||
| 34 | In order to resolve the issue, I had to ensure the `/var/lib/nginx` | |
| 35 | directory is owned by Nginx. Mine was owned by the `www` user and Nginx was not | |
| 36 | able to read or write files within that directory. This prevented Nginx from | |
| 37 | caching temporary files. | |
| 38 | ||
| 39 | ```sh | |
| 40 | # Alpine Linux | |
| 41 | doas chown -R nginx:nginx /var/lib/nginx | |
| 42 | ||
| 43 | # Other Distros | |
| 44 | sudo chown -R nginx:nginx /var/lib/nginx | |
| 45 | ``` | |
| 46 | ||
| 47 | You *may* also be able to change the `proxy_temp_path` in your Nginx config, but | |
| 48 | I did not try this. Here's a suggestion I found online that may work if the | |
| 49 | above solution does not: | |
| 50 | ||
| 51 | ```sh | |
| 52 | nano /etc/nginx/http.d/example.com.conf | |
| 53 | ``` | |
| 54 | ||
| 55 | ```conf | |
| 56 | server { | |
| 57 | ... | |
| 58 | ||
| 59 | # Set the proxy_temp_path to your preference, make sure it's owned by the | |
| 60 | # `nginx` user | |
| 61 | proxy_temp_path /tmp; | |
| 62 | ||
| 63 | ... | |
| 64 | } | |
| 65 | ``` | |
| 66 | ||
| 67 | Finally, restart Nginx and your server should be able to cache temporary files | |
| 68 | again. | |
| 69 | ||
| 70 | ```sh | |
| 71 | # Alpine Linux (OpenRC) | |
| 72 | doas rc-service nginx restart | |
| 73 | ||
| 74 | # Other Distros (systemd) | |
| 75 | sudo systemctl restart nginx | |
| 76 | ``` | |
blog/2022-11-27-server-build.org added +144
| @@ -0,0 +1,144 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-11-27 | |
| 3 | title = "Building a Custom Rack-Mount Server" | |
| 4 | description = "I finally built my own custom rack-mounted server after years of indecision." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## The Dilemma | |
| 8 | ||
| 9 | For years, I have been using desktops and a Raspberry Pi as the backbone of my | |
| 10 | homelab. I have always wanted to move toward a single dedicated server that | |
| 11 | could handle all of my tasks, but was often put off by the complexity of the | |
| 12 | choices (and financial roadblocks at some times). | |
| 13 | ||
| 14 | However, after purchasing a small server rack this past year, I have been | |
| 15 | researching to see what kind of rack-mounted server I could buy. I initially | |
| 16 | bought a Dell R720XD loaded up with everything I could ever want in a server - | |
| 17 | but I did not care for it. It was far too loud, and the age of the CPU/RAM was | |
| 18 | something I wanted to improve upon. | |
| 19 | ||
| 20 | After returning the R720XD, I decided that I wanted to build my own server with | |
| 21 | modern, consumer-grade PC components. This time, I am very happy with the | |
| 22 | results of my server. | |
| 23 | ||
| 24 | ## Components | |
| 25 | ||
| 26 | I'll start by listing all the components I used for this server build: | |
| 27 | ||
| 28 | - **Case**: [Rosewill RSV-R4100U 4U Server Chassis Rackmount | |
| 29 | Case](https://www.rosewill.com/rosewill-rsv-r4100u-black/p/9SIA072GJ92825) | |
| 30 | - **Motherboard**: [NZXT B550](https://nzxt.com/product/n7-b550) | |
| 31 | - **CPU**: AMD Ryzen 7 5700G with Radeon Graphics | |
| 32 | - **GPU**: N/A - I specifically chose one of the few AMD CPUs that support | |
| 33 | onboard graphics. | |
| 34 | - **RAM**: 64GB RAM (2x32GB) _Max of 128GB RAM on this motherboard_ | |
| 35 | - **Boot Drive**: Western Digital 500GB M.2 NVME SSD | |
| 36 | - **HDD Bay**: | |
| 37 | - 10TB WD White _(shucked, moved from previous server)_ | |
| 38 | - 8TB WD White _(shucked, moved from previous server)_ | |
| 39 | - 2 x 8TB WD Red Plus _(Black Friday lined up perfectly with this build, so I | |
| 40 | grabbed two of these)_ | |
| 41 | - **PSU**: Corsair RM850 PSU | |
| 42 | - **Extras**: | |
| 43 | - Corsair TM3Q Thermal Paste | |
| 44 | - Noctua 120mm fan _(replacement for front case fan)_ | |
| 45 | - 2 x Noctua 80mm fans _(replacement for rear case fans)_ | |
| 46 | - CableMatters 6Gbps SATA Cables | |
| 47 | ||
| 48 | ## Building the Server | |
| 49 | ||
| 50 | This took quite a while for me to build (in my opinion of time), totaling | |
| 51 | around 3 hours from start to finish. The case has some peculiar construction, so | |
| 52 | you have to completely remove the ODD & HDD cages to install the motherboard and | |
| 53 | other components first. | |
| 54 | ||
| 55 | Now, I've never built a computer of any kind before, so I was quite | |
| 56 | nervous. Personally, the only challenging part was getting the CPU cooler to | |
| 57 | screw into the motherboard without sliding the thermal paste around too much | |
| 58 | underneath. I'm still not entirely sure if I did a great job of it, but | |
| 59 | nothing's broken yet. | |
| 60 | ||
| 61 | The main components were all fine and simple. | |
| 62 | However, installing the hard drives is slightly tedious as I need to power | |
| 63 | off the server and completely unscrew the HDD cage to install or remove any | |
| 64 | drives. | |
| 65 | Additionally, the drives are screwed directly into the metal cage with small | |
| 66 | screws, which are quite a bit different from the HDD trays I'm used to in | |
| 67 | other machines. | |
| 68 | ||
| 69 | Seeing that the cases with hot-swap bays were 3-4x the price, I'm okay dealing | |
| 70 | with the tedium of removing the cage to install new drives. | |
| 71 | ||
| 72 | ## Software | |
| 73 | ||
| 74 | I'm not going to dive into the software as I have done so in other recent posts. | |
| 75 | However, I wanted to note that I am using Alpine Linux on this server and | |
| 76 | hosting most services inside Docker. No virtual machines (VMs) and very few | |
| 77 | bare-metal services. | |
| 78 | ||
| 79 | ## The Results | |
| 80 | ||
| 81 | How did my build turn out? Well, after migrating my other servers and their | |
| 82 | services over, I found that my server is blazing fast. The heaviest of my | |
| 83 | applications, Plex, is handled with ease. Even 4k streaming seems to be | |
| 84 | effortless. | |
| 85 | ||
| 86 | I am very happy with the results and will likely continue to improve on this | |
| 87 | server as the years go by rather than buying another used server online. | |
| 88 | ||
| 89 | ### Mistakes I Made | |
| 90 | ||
| 91 | This post wouldn't be complete unless I wrote about the mistakes I made while | |
| 92 | building. The only real mistake I made beyond a "whoops I dropped a screw" | |
| 93 | related to airflow and fan direction. | |
| 94 | ||
| 95 | While installing the two new hard drives that showed up on 2022-11-30 and | |
| 96 | getting ready to install the case in my rack, I noticed that the hard drive | |
| 97 | temperatures were quite high. | |
| 98 | ||
| 99 | I used the `smartctl` command for each of my drives (`/dev/sda` through | |
| 100 | `/dev/sdd`): | |
| 101 | ||
| 102 | ```sh | |
| 103 | doas smartctl -a /dev/sda | grep Temperature_Celsius | |
| 104 | ``` | |
| 105 | ||
| 106 | The results were unusual - all four drives were idling at ~44-46 degrees | |
| 107 | Celsius. The only drive that was cooler was my 10TB drive, which was at 38 | |
| 108 | degrees Celsius. I noted that this 10TB drive was also closest to the case fan. | |
| 109 | ||
| 110 | ```sh | |
| 111 | ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE | |
| 112 | # /dev/sda | |
| 113 | 194 Temperature_Celsius 0x0002 147 147 000 Old_age Always - 44 (Min/Max 22/46) | |
| 114 | # /dev/sdb | |
| 115 | 194 Temperature_Celsius 0x0002 141 141 000 Old_age Always - 46 (Min/Max 21/48) | |
| 116 | # /dev/sdc | |
| 117 | 194 Temperature_Celsius 0x0002 144 144 000 Old_age Always - 45 (Min/Max 19/61) | |
| 118 | # /dev/sdd | |
| 119 | 194 Temperature_Celsius 0x0002 171 171 000 Old_age Always - 38 (Min/Max 14/56) | |
| 120 | ``` | |
| 121 | ||
| 122 | After looking to see if I could fit more fans into the case, I noticed that the | |
| 123 | 120mm fan used for intake from the front of the case was actually pushing air | |
| 124 | out of the case by mistake. This fan sits right in front of the hard drive bay. | |
| 125 | ||
| 126 | Once I flipped the fan around to act as an intake fan, the temperatures dropped | |
| 127 | immediately! They are now idling at ~31-33 degrees Celsius. A single fan | |
| 128 | spinning the wrong way caused my drives to idle 10-15 degrees higher than they | |
| 129 | should have. | |
| 130 | ||
| 131 | ```sh | |
| 132 | ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE | |
| 133 | # /dev/sda | |
| 134 | 194 Temperature_Celsius 0x0002 209 209 000 Old_age Always - 31 (Min/Max 14/56) | |
| 135 | # /dev/sdb | |
| 136 | 194 Temperature_Celsius 0x0002 196 196 000 Old_age Always - 33 (Min/Max 19/61) | |
| 137 | # /dev/sdc | |
| 138 | 194 Temperature_Celsius 0x0002 203 203 000 Old_age Always - 32 (Min/Max 21/48) | |
| 139 | # /dev/sdd | |
| 140 | 194 Temperature_Celsius 0x0002 196 196 000 Old_age Always - 33 (Min/Max 22/46) | |
| 141 | ``` | |
| 142 | ||
| 143 | This was a silly error to make, but I'm glad I found it today before I screwed | |
| 144 | the case into the rack and made things a lot more tedious to fix. | |
blog/2022-11-29-nginx-referrer-ban-list.org added +133
| @@ -0,0 +1,133 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-11-29 | |
| 3 | title = "Creating a Referrer Ban List on Nginx" | |
| 4 | description = "A quick explanation detailing my own way of banning referral domains on Nginx." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Creating the Ban List | |
| 8 | ||
| 9 | In order to ban list referral domains or websites with Nginx, you need to | |
| 10 | create a ban list file. | |
| 11 | The file below will accept regexes for different domains or websites you | |
| 12 | wish to block. | |
| 13 | ||
| 14 | First, create the file in your nginx directory: | |
| 15 | ||
| 16 | ```sh | |
| 17 | doas nano /etc/nginx/banlist.conf | |
| 18 | ``` | |
| 19 | ||
| 20 | Next, paste the following contents in and fill out the regexes with whichever | |
| 21 | domains you're blocking. | |
| 22 | ||
| 23 | ```conf | |
| 24 | # /etc/nginx/banlist.conf | |
| 25 | ||
| 26 | map $http_referer $bad_referer { | |
| 27 | hostnames; | |
| 28 | ||
| 29 | default 0; | |
| 30 | ||
| 31 | # Put regexes for undesired referrers here | |
| 32 | "~news.ycombinator.com" 1; | |
| 33 | } | |
| 34 | ``` | |
| 35 | ||
| 36 | ## Configuring Nginx | |
| 37 | ||
| 38 | In order for the ban list to work, Nginx needs to know it exists and how to | |
| 39 | handle it. For this, edit the `nginx.conf` file. | |
| 40 | ||
| 41 | ```sh | |
| 42 | doas nano /etc/nginx/nginx.conf | |
| 43 | ``` | |
| 44 | ||
| 45 | Within this file, find the `http` block and add your ban list file location to | |
| 46 | the end of the block. | |
| 47 | ||
| 48 | ```conf | |
| 49 | # /etc/nginx/nginx.conf | |
| 50 | ||
| 51 | http { | |
| 52 | ... | |
| 53 | ||
| 54 | # Include ban list | |
| 55 | include /etc/nginx/banlist.conf; | |
| 56 | } | |
| 57 | ``` | |
| 58 | ||
| 59 | ## Enabling the Ban List | |
| 60 | ||
| 61 | Finally, we need to take action when a bad referral site is found. To do so, | |
| 62 | edit the configuration file for your website. For example, I have all website | |
| 63 | configuration files in the `http.d` directory. You may have them in the | |
| 64 | `sites-available` directory on some distributions. | |
| 65 | ||
| 66 | ```sh | |
| 67 | doas nano /etc/nginx/http.d/example.com.conf | |
| 68 | ``` | |
| 69 | ||
| 70 | Within each website's configuration file, edit the `server` blocks that are | |
| 71 | listening to ports 80 and 443 and create a check for the `$bad_referrer` | |
| 72 | variable we created in the ban list file. | |
| 73 | ||
| 74 | If a matching site is found, you can return any [HTTP Status | |
| 75 | Code](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes) you want. Code | |
| 76 | 403 (Forbidden) is logical in this case since you are preventing a client | |
| 77 | connection due to a banned domain. | |
| 78 | ||
| 79 | ```conf | |
| 80 | server { | |
| 81 | ... | |
| 82 | ||
| 83 | # If a referral site is banned, return an error | |
| 84 | if ($bad_referer) { | |
| 85 | return 403; | |
| 86 | } | |
| 87 | ||
| 88 | ... | |
| 89 | } | |
| 90 | ``` | |
| 91 | ||
| 92 | ## Restart Nginx | |
| 93 | ||
| 94 | Lastly, restart Nginx to enable all changes made. | |
| 95 | ||
| 96 | ```sh | |
| 97 | doas rc-service nginx restart | |
| 98 | ``` | |
| 99 | ||
| 100 | ## Testing Results | |
| 101 | ||
| 102 | In order to test the results, let's curl the contents of our site. To start, | |
| 103 | I'll curl the site normally: | |
| 104 | ||
| 105 | ```sh | |
| 106 | curl https://0x4b1d.org | |
| 107 | ``` | |
| 108 | ||
| 109 | The HTML contents of the page come back successfully: | |
| 110 | ||
| 111 | ```html | |
| 112 | <!doctype html>...</html> | |
| 113 | ``` | |
| 114 | ||
| 115 | Next, let's include a banned referrer: | |
| 116 | ||
| 117 | ```sh | |
| 118 | curl --referer https://news.ycombinator.com https://0x4b1d.org | |
| 119 | ``` | |
| 120 | ||
| 121 | This time, I'm met with a 403 Forbidden response page. That means we are | |
| 122 | successful and any clients being referred from a banned domain will be met | |
| 123 | with this same response code. | |
| 124 | ||
| 125 | ```html | |
| 126 | <html> | |
| 127 | <head><title>403 Forbidden</title></head> | |
| 128 | <body> | |
| 129 | <center><h1>403 Forbidden</h1></center> | |
| 130 | <hr><center>nginx</center> | |
| 131 | </body> | |
| 132 | </html> | |
| 133 | ``` | |
blog/2022-12-01-nginx-compression.org added +74
| @@ -0,0 +1,74 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-12-01 | |
| 3 | title = "Enable GZIP Compression on Nginx" | |
| 4 | desciption = "Enable gzip compression on Nginx to server small text-based resources faster." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Text Compression | |
| 8 | ||
| 9 | Text compression allows a web server to serve text-based resources faster than | |
| 10 | uncompressed data. This can speed up things like First Contentful Paint, Tie to | |
| 11 | Interactive, and Speed Index. | |
| 12 | ||
| 13 | ## Enable Nginx Compression with gzip | |
| 14 | ||
| 15 | In order to enable text compression on Nginx, we need to enable it within the | |
| 16 | configuration file: | |
| 17 | ||
| 18 | ```sh | |
| 19 | nano /etc/nginx/nginx.conf | |
| 20 | ``` | |
| 21 | ||
| 22 | Within the `http` block, find the section that shows something like the block | |
| 23 | below. This is the default gzip configuration I found in my `nginx.conf` file on | |
| 24 | Alpine Linux 3.17. Yours may look slightly different, just make sure that | |
| 25 | you're not creating any duplicate gzip options. | |
| 26 | ||
| 27 | ```conf | |
| 28 | # Enable gzipping of responses. | |
| 29 | #gzip on; | |
| 30 | ||
| 31 | # Set the Vary HTTP header as defined in the RFC 2616. Default is 'off'. | |
| 32 | gzip_vary on; | |
| 33 | ``` | |
| 34 | ||
| 35 | Remove the default gzip lines and replace them with the following: | |
| 36 | ||
| 37 | ```conf | |
| 38 | # Enable gzipping of responses. | |
| 39 | gzip on; | |
| 40 | gzip_vary on; | |
| 41 | gzip_min_length 10240; | |
| 42 | gzip_proxied expired no-cache no-store private auth; | |
| 43 | gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml; | |
| 44 | gzip_disable "MSIE [1-6]\."; | |
| 45 | ``` | |
| 46 | ||
| 47 | ## Explanations of ngx_http_gzip_module Options | |
| 48 | ||
| 49 | Each of the lines above enables a different aspect of the gzip response for | |
| 50 | Nginx. Here are the full explanations: | |
| 51 | ||
| 52 | ||
| 53 | - `gzip` – Enables or disables gzipping of responses. | |
| 54 | - `gzip_vary` – Enables or disables inserting the “Vary: Accept-Encoding” | |
| 55 | response header field if the directives gzip, gzip_static, or gunzip are active. | |
| 56 | - `gzip_min_length` – Sets the minimum length of a response that will be | |
| 57 | gzipped. The length is determined only from the “Content-Length” response header field. | |
| 58 | - `gzip_proxied` – Enables or disables gzipping of responses for proxied | |
| 59 | requests depending on the request and response. The fact that the request is | |
| 60 | proxied is determined by the presence of the “Via” request header field. | |
| 61 | - `gzip_types` – Enables gzipping of responses for the specified MIME types in | |
| 62 | addition to “text/html”. The special value “*” matches any MIME type (0.8.29). | |
| 63 | Responses with the “text/html” type are always compressed. | |
| 64 | - `gzip_disable` – Disables gzipping of responses for requests | |
| 65 | with “User-Agent” header fields matching any of the specified regular | |
| 66 | expressions. | |
| 67 | - The special mask “msie6” (0.7.12) corresponds to the regular expression “MSIE | |
| 68 | [4-6]\.”, but works faster. Starting from version 0.8.11, “MSIE 6.0; ... SV1” is | |
| 69 | excluded from this mask. | |
| 70 | ||
| 71 | More information on these directives and their options can be found on the | |
| 72 | [Module | |
| 73 | ngx_http_gzip_module](https://nginx.org/en/docs/http/ngx_http_gzip_module.html) | |
| 74 | page in Nginx's documentation. | |
blog/2022-12-07-nginx-wildcard-redirect.org added +119
| @@ -0,0 +1,119 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-12-07 | |
| 3 | title = "Redirect Nginx Subdomains & Trailing Content with Regex" | |
| 4 | description= "You can easily redirect Nginx traffic from subdomains and TLDs to a new domain while preserving trailing content." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Problem | |
| 8 | ||
| 9 | I recently migrated domains and replaced the | |
| 10 | old webpage with a simple info page with instructions to users on how | |
| 11 | to edit their bookmarks and URLs to get to the page they were seeking. | |
| 12 | ||
| 13 | This was not ideal as it left the work up to the user and may have caused | |
| 14 | friction for users who accessed my RSS feed. | |
| 15 | ||
| 16 | ## Solution | |
| 17 | ||
| 18 | Instead, I finally found a solution that allows me to redirect both subdomains | |
| 19 | AND trailing content. For example, both of these URLs now redirect properly | |
| 20 | using the logic I'll explain below: | |
| 21 | ||
| 22 | ```txt | |
| 23 | # Example 1 - Simple base domain redirect with trailing content | |
| 24 | https://domain1.com/blog/alpine-linux/ -> https://domain2.com/blog/alpine-linux/ | |
| 25 | ||
| 26 | # Example 2 - Complex redirect with both a subdomain and trailing content | |
| 27 | https://libreddit.domain1.com/r/history/comments/7z8cbg/new_discovery_mode_turns_video_game_assassins/ | |
| 28 | -> | |
| 29 | https://libreddit.domain2.com/r/history/comments/7z8cbg/new_discovery_mode_turns_video_game_assassins/ | |
| 30 | ``` | |
| 31 | ||
| 32 | Go ahead, try the URLs if you want to test them. | |
| 33 | ||
| 34 | ### Nginx Config | |
| 35 | ||
| 36 | To make this possible. I needed to configure a proper redirect scheme in my | |
| 37 | Nginx configuration. | |
| 38 | ||
| 39 | ```sh | |
| 40 | doas nano /etc/nginx/http.d/domain1.conf | |
| 41 | ``` | |
| 42 | ||
| 43 | Within this file, I had one block configured to redirect HTTP requests to HTTPS | |
| 44 | for the base domain and all subdomains. | |
| 45 | ||
| 46 | ```conf | |
| 47 | server { | |
| 48 | listen [::]:80; | |
| 49 | listen 80; | |
| 50 | server_name domain1.com *.domain1.com; | |
| 51 | ||
| 52 | if ($host = domain1.com) { | |
| 53 | return 301 https://$host$request_uri; | |
| 54 | } | |
| 55 | ||
| 56 | if ($host = *.domain1.com) { | |
| 57 | return 301 https://$host$request_uri; | |
| 58 | } | |
| 59 | ||
| 60 | return 404; | |
| 61 | } | |
| 62 | ``` | |
| 63 | ||
| 64 | For the base domain, I have another `server` block dedicated to redirecting all | |
| 65 | base domain requests. You can see that the `rewrite` line is instructing Nginx | |
| 66 | to gather all trailing content and append it to the new `domain2.com` URL. | |
| 67 | ||
| 68 | ```conf | |
| 69 | server { | |
| 70 | listen [::]:443 ssl http2; | |
| 71 | listen 443 ssl http2; | |
| 72 | ||
| 73 | server_name domain1.com; | |
| 74 | ||
| 75 | rewrite ^/(.*)$ https://domain2.com/$1 permanent; | |
| 76 | ||
| 77 | ssl_certificate /etc/letsencrypt/live/domain1.com/fullchain.pem; | |
| 78 | ssl_certificate_key /etc/letsencrypt/live/domain1.com/privkey.pem; | |
| 79 | } | |
| 80 | ``` | |
| 81 | ||
| 82 | Finally, the tricky part is figuring out how to tell Nginx to redirect while | |
| 83 | keeping both a subdomain and trailing content intact. I found that the easiest | |
| 84 | way to do this is to give it a `server` block of its own. | |
| 85 | ||
| 86 | Within this block, we need to do some regex on the `server_name` line before we | |
| 87 | can rewrite anything. This creates a variable called `subdomain`. | |
| 88 | ||
| 89 | Once the server gets to the `rewrite` line, it pulls the `subdomain` variable | |
| 90 | from above and uses it on the new `domain2.com` domain before appending the | |
| 91 | trailing content (`$request_uri`). | |
| 92 | ||
| 93 | ```conf | |
| 94 | server { | |
| 95 | listen [::]:443 ssl http2; | |
| 96 | listen 443 ssl http2; | |
| 97 | ||
| 98 | server_name ~^(?<subdomain>\w+)\.domain1\.com$; | |
| 99 | ||
| 100 | rewrite ^ https://$subdomain.domain2.com$request_uri permanent; | |
| 101 | ||
| 102 | ssl_certificate /etc/letsencrypt/live/domain1.com/fullchain.pem; | |
| 103 | ssl_certificate_key /etc/letsencrypt/live/domain1.com/privkey.pem; | |
| 104 | } | |
| 105 | ``` | |
| 106 | ||
| 107 | That's all there is to it. With this, I simply restarted Nginx and watched the | |
| 108 | redirections work in-action. | |
| 109 | ||
| 110 | ```sh | |
| 111 | doas rc-service nginx restart | |
| 112 | ``` | |
| 113 | ||
| 114 | ||
| 115 | Looking back on it, I wish I had done this sooner. Who knows how many people | |
| 116 | went looking for my sites or bookmarks and gave up when they saw the redirect | |
| 117 | instructions page. | |
| 118 | ||
| 119 | Oh well, it's done now. Live and learn. | |
blog/2022-12-17-st.org added +89
| @@ -0,0 +1,89 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-12-17 | |
| 3 | title = "Simple Terminal" | |
| 4 | description = "An easy guide on how to build st on Fedora Workstation." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## st | |
| 8 | ||
| 9 | [st](https://st.suckless.org) standards for Simple Terminal, a simple terminal | |
| 10 | implementation for X made by the [suckless](https://suckless.org) team. | |
| 11 | ||
| 12 | This post walks through the dependencies needed and process to build and install | |
| 13 | `st` on Fedora Workstation. | |
| 14 | ||
| 15 | ### Obtain Files | |
| 16 | ||
| 17 | To start, obtain the source files for `st` via `git clone`. | |
| 18 | ||
| 19 | ```sh | |
| 20 | mkdir ~/suckless && cd ~/suckless | |
| 21 | git clone https://git.suckless.org/st && cd st | |
| 22 | ``` | |
| 23 | ||
| 24 | ### Dependencies | |
| 25 | ||
| 26 | Once you have the files and are in the `st` directory, ensure the following | |
| 27 | packages are installed. | |
| 28 | ||
| 29 | ```sh | |
| 30 | sudo dnf update && sudo dnf upgrade | |
| 31 | sudo dnf install gcc patch libX11-devel libXft-devel | |
| 32 | ``` | |
| 33 | ||
| 34 | ### Building | |
| 35 | ||
| 36 | Before building, ensure that you read the README file. | |
| 37 | ||
| 38 | ```sh | |
| 39 | cat README | |
| 40 | ``` | |
| 41 | ||
| 42 | Once you've read the instructions, open the `config.mk` file and ensure it | |
| 43 | matches your setup. If you're not sure, leave the default options within the | |
| 44 | file. | |
| 45 | ||
| 46 | Finally, you can build `st` with the following command. Ensure you run as root | |
| 47 | (e.g., `sudo`) or else you may not end up with a usable application file. | |
| 48 | ||
| 49 | ```sh | |
| 50 | sudo make clean install | |
| 51 | ``` | |
| 52 | ||
| 53 | ### Customization (Patches) | |
| 54 | ||
| 55 | Note that customizing `st` requires you to modify the source files or to | |
| 56 | download one of the [available patches](https://st.suckless.org/patches/) for | |
| 57 | suckless.org. | |
| 58 | ||
| 59 | If you've already installed `st` and want to customize or install a patch, start | |
| 60 | by uninstalling the current program. | |
| 61 | ||
| 62 | ```sh | |
| 63 | cd ~/suckless/st | |
| 64 | sudo make uninstall | |
| 65 | ``` | |
| 66 | ||
| 67 | Next, grab the `<path>.diff` file from the page of the patch you chose. For | |
| 68 | example, I will be using the | |
| 69 | [defaultfontsize](https://st.suckless.org/patches/defaultfontsize/) patch in the below example. | |
| 70 | ||
| 71 | ```sh | |
| 72 | wget https://st.suckless.org/patches/defaultfontsize/st-defaultfontsize-20210225-4ef0cbd.diff | |
| 73 | ``` | |
| 74 | ||
| 75 | Once the file is downloaded inside the `st` folder, apply the patch and | |
| 76 | re-install the program. You may need to install the `patch` command if you don't | |
| 77 | have it installed already (you should have installed it above). | |
| 78 | ||
| 79 | ```sh | |
| 80 | patch -i st-defaultfontsize-20210225-4ef0cbd.diff | |
| 81 | sudo make clean install | |
| 82 | ``` | |
| 83 | ||
| 84 | Once installed, you can use the default font size patch to launch `st` with any | |
| 85 | font size you wish: | |
| 86 | ||
| 87 | ```sh | |
| 88 | st -z 16 | |
| 89 | ``` | |
blog/2022-12-23-alpine-desktop.org added +266
| @@ -0,0 +1,266 @@ | ||
| 1 | +++ | |
| 2 | date = 2022-12-23 | |
| 3 | title = "Alpine Linux as a Desktop OS" | |
| 4 | description = "Documenting my steps to a useable Alpine Linux desktop experience." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Isn't Alpine Linux for Servers? | |
| 8 | ||
| 9 | This is a question I see a lot when people are presented with an example of | |
| 10 | Alpine Linux running as a desktop OS. | |
| 11 | ||
| 12 | While Alpine is small, fast, and minimal, that doesn't stop it from functioning | |
| 13 | at a productive level for desktop users. | |
| 14 | ||
| 15 | This post is documentation of how I installed and modified Alpine Linux to | |
| 16 | become my daily desktop OS. | |
| 17 | ||
| 18 | ## Installation | |
| 19 | ||
| 20 | Note that I cover the installation of Alpine Linux in my other post, so I | |
| 21 | won't repeat it here: [Alpine Linux: My New Server OS](/blog/alpine-linux/). | |
| 22 | ||
| 23 | Basically, get a bootable USB or whatever you prefer with Alpine on it, boot the | |
| 24 | ISO, and run the setup script. | |
| 25 | ||
| 26 | ```sh | |
| 27 | setup-alpine | |
| 28 | ``` | |
| 29 | ||
| 30 | Once you have gone through all the options and installer finishes without | |
| 31 | errors, reboot. | |
| 32 | ||
| 33 | ```sh | |
| 34 | reboot | |
| 35 | ``` | |
| 36 | ||
| 37 | ## Initial Setup | |
| 38 | ||
| 39 | Once Alpine is installed and the machine has rebooted, login is as root | |
| 40 | initially or `su` to root once you log in as your user. | |
| 41 | From here, you should start by updating and upgrading the system in case the | |
| 42 | ISO was not fully up-to-date. | |
| 43 | ||
| 44 | ```sh | |
| 45 | # Update and upgrade system | |
| 46 | apk -U update && apk -U upgrade | |
| 47 | ||
| 48 | # Add an editor so we can enable the community repository | |
| 49 | apk add nano | |
| 50 | ``` | |
| 51 | ||
| 52 | You need to uncomment the `community` repository for your version of Alpine | |
| 53 | Linux. | |
| 54 | ||
| 55 | For v3.17, the `repositories` file should look like this: | |
| 56 | ||
| 57 | ```sh | |
| 58 | nano /etc/apk/repositories | |
| 59 | ``` | |
| 60 | ||
| 61 | ```conf | |
| 62 | #/media/sda/apks | |
| 63 | http://mirrors.gigenet.com/alpinelinux/v3.17/main | |
| 64 | http://mirrors.gigenet.com/alpinelinux/v3.17/community | |
| 65 | #http://mirrors.gigenet.com/alpinelinux/edge/main | |
| 66 | #http://mirrors.gigenet.com/alpinelinux/edge/community | |
| 67 | #http://mirrors.gigenet.com/alpinelinux/edge/testing | |
| 68 | ``` | |
| 69 | ||
| 70 | ```sh | |
| 71 | # Add the rest of your packages | |
| 72 | apk add linux-firmware iwd doas git curl wget | |
| 73 | ||
| 74 | # Add yourself to the wheel group so you can use the doas command | |
| 75 | adduser $USER wheel | |
| 76 | ``` | |
| 77 | ||
| 78 | ## Window Manager (Desktop) | |
| 79 | ||
| 80 | The [Sway installation guide](https://wiki.alpinelinux.org/wiki/Sway) has | |
| 81 | everything you need to get Sway working on Alpine. | |
| 82 | ||
| 83 | However, I'll include a brief list of the commands I ran and their purpose for | |
| 84 | posterity here. | |
| 85 | ||
| 86 | ```sh | |
| 87 | # Add eudev and set it up | |
| 88 | apk add eudev | |
| 89 | setup-devd udev | |
| 90 | ||
| 91 | # Since I have Radeon graphics, I need the following packages | |
| 92 | apk add mesa-dri-gallium mesa-va-gallium | |
| 93 | ||
| 94 | # Add user to applicable groups | |
| 95 | adduser $USER input | |
| 96 | adduser $USER video | |
| 97 | ||
| 98 | # Add a font package | |
| 99 | apk add ttf-dejavu | |
| 100 | ||
| 101 | # Add the seatd daemon | |
| 102 | apk add seatd | |
| 103 | rc-update add seatd | |
| 104 | rc-service seatd start | |
| 105 | ||
| 106 | # Add user to seat group | |
| 107 | adduser $USER seat | |
| 108 | ||
| 109 | # Add elogind | |
| 110 | apk add elogind polkit-elogind | |
| 111 | rc-update add elogind | |
| 112 | rc-service elogind start | |
| 113 | ||
| 114 | # Finally, add sway and dependencies | |
| 115 | apk add sway sway-doc | |
| 116 | apk add \ # Install optional dependencies: | |
| 117 | xwayland \ # recommended for compatibility reasons | |
| 118 | foot \ # default terminal emulator | |
| 119 | bemenu \ # wayland menu | |
| 120 | swaylock swaylockd \ # lockscreen tool | |
| 121 | swaybg \ # wallpaper daemon | |
| 122 | swayidle # idle management (DPMS) daemon | |
| 123 | ``` | |
| 124 | ||
| 125 | Once you have the packages installed and set-up, you need to export the | |
| 126 | `XDG_RUNTIME_DIR` upon login. To do this, edit your `.profile` file. | |
| 127 | ||
| 128 | If you use another shell, such as `zsh`, you need to edit that shell's profile | |
| 129 | (e.g., `~/.zprofile`)! | |
| 130 | ||
| 131 | ```sh | |
| 132 | nano ~/.profile | |
| 133 | ``` | |
| 134 | ||
| 135 | Within the file, paste this: | |
| 136 | ||
| 137 | ```sh | |
| 138 | if test -z "${XDG_RUNTIME_DIR}"; then | |
| 139 | export XDG_RUNTIME_DIR=/tmp/$(id -u)-runtime-dir | |
| 140 | if ! test -d "${XDG_RUNTIME_DIR}"; then | |
| 141 | mkdir "${XDG_RUNTIME_DIR}" | |
| 142 | chmod 0700 "${XDG_RUNTIME_DIR}" | |
| 143 | fi | |
| 144 | fi | |
| 145 | ``` | |
| 146 | ||
| 147 | Once that's complete, you can launch Sway manually. | |
| 148 | ||
| 149 | ```sh | |
| 150 | dbus-run-session -- sway | |
| 151 | ``` | |
| 152 | ||
| 153 | ## Personal Touches | |
| 154 | ||
| 155 | I also added the following packages, per my personal preferences and situation. | |
| 156 | ||
| 157 | ```sh | |
| 158 | doas apk add brightnessctl \ # Brightness controller | |
| 159 | zsh \ # Shell | |
| 160 | firefox \ # Browser | |
| 161 | syncthing \ # File sync service | |
| 162 | wireguard-tools \ # Wireguard VPN | |
| 163 | gomuks \ # CLI Matrix client | |
| 164 | neomutt \ # CLI email client | |
| 165 | thunderbird \ # GUI email client | |
| 166 | gnupg # GPG key manager | |
| 167 | ``` | |
| 168 | ||
| 169 | From here, I use my Syncthing storage to pull all the configuration files I | |
| 170 | stored from prior desktops, such as [my | |
| 171 | dotfiles](https://git.sr.ht/~cmc/dotfiles). | |
| 172 | ||
| 173 | ## Resolving Issues | |
| 174 | ||
| 175 | ### WiFi Issues | |
| 176 | ||
| 177 | I initially tried to set up my Wi-Fi the standard way with `iwd`, but it didn't | |
| 178 | work. | |
| 179 | ||
| 180 | Here is what I initially tried (I did all of this as `root`): | |
| 181 | ||
| 182 | ```sh | |
| 183 | apk add iwd | |
| 184 | rc-service iwd start | |
| 185 | iwctl station wlan0 connect <SSID> # This will prompt for the password | |
| 186 | rc-update add iwd boot && rc-update add dbus boot | |
| 187 | ``` | |
| 188 | ||
| 189 | Then, I added the Wi-Fi entry to the bottom of the networking interface file: | |
| 190 | ||
| 191 | ```sh | |
| 192 | nano /etc/network/interfaces | |
| 193 | ``` | |
| 194 | ||
| 195 | ```conf | |
| 196 | auto wlan0 | |
| 197 | iface wlan0 inet dhcp | |
| 198 | ``` | |
| 199 | ||
| 200 | Finally, restart the networking service: | |
| 201 | ||
| 202 | ```sh | |
| 203 | rc-service networking restart | |
| 204 | ``` | |
| 205 | ||
| 206 | My Wi-Fi interface would receive an IP address from the router, but it could | |
| 207 | not ping anything in the network. | |
| 208 | To solve the Wi-Fi issues, I originally upgraded to Alpine's `edge` | |
| 209 | repositories, which was unnecessary. | |
| 210 | ||
| 211 | Really, the solution was to enable the `NameResolvingService=resolvconf` in | |
| 212 | `/etc/iwd/main.conf`. | |
| 213 | ||
| 214 | ```sh | |
| 215 | doas nano /etc/iwd/main.conf | |
| 216 | ``` | |
| 217 | ||
| 218 | ```conf | |
| 219 | [Network] | |
| 220 | ||
| 221 | NameResolvingService=resolvconf | |
| 222 | ``` | |
| 223 | ||
| 224 | Once I finished this process, my Wi-Fi is working flawlessly. | |
| 225 | ||
| 226 | ### Sound Issues | |
| 227 | ||
| 228 | Same as with the Wi-Fi, I had no sound and could not control the mute/unmute or | |
| 229 | volume buttons on my laptop. | |
| 230 | ||
| 231 | To resolve this, I installed | |
| 232 | [pipewire](https://wiki.alpinelinux.org/wiki/PipeWire). | |
| 233 | ||
| 234 | ```sh | |
| 235 | # Add your user to the following groups | |
| 236 | addgroup $USER audio | |
| 237 | addgroup $USER video | |
| 238 | ||
| 239 | # Install pipewire and other useful packages | |
| 240 | apk add pipewire wireplumber pipewire-pulse pipewire-jack pipewire-alsa | |
| 241 | ``` | |
| 242 | ||
| 243 | Finally, I needed to add `/usr/libexec/pipewire-launcher` to my | |
| 244 | `.config/sway/config` file so that Pipewire would run every time I launched | |
| 245 | sway. | |
| 246 | ||
| 247 | ```sh | |
| 248 | nano ~/.config/sway/config | |
| 249 | ``` | |
| 250 | ||
| 251 | ```conf | |
| 252 | # Run pipewire audio server | |
| 253 | exec /usr/libexec/pipewire-launcher | |
| 254 | ||
| 255 | # Example audio button controls | |
| 256 | bindsym XF86AudioRaiseVolume exec --no-startup-id pactl set-sink-volume @DEFAULT_SINK@ +5% | |
| 257 | bindsym XF86AudioLowerVolume exec --no-startup-id pactl set-sink-volume @DEFAULT_SINK@ -5% | |
| 258 | bindsym XF86AudioMute exec --no-startup-id pactl set-sink-mute @DEFAULT_SINK@ toggle | |
| 259 | bindsym XF86AudioMicMute exec --no-startup-id pactl set-source-mute @DEFAULT_SOURCE@ toggle | |
| 260 | ``` | |
| 261 | ||
| 262 | Note that I do not use bluetooth or screen sharing, so I won't cover those | |
| 263 | options in this post. | |
| 264 | ||
| 265 | Other than these issues, I have a working Alpine desktop. No other complaints | |
| 266 | thus far! | |
blog/2023-01-03-recent-website-changes.org added +82
| @@ -0,0 +1,82 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-01-03 | |
| 3 | title = "Recent Website Changes" | |
| 4 | description = "A quick review on this website's recent changes." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## The State of This Website | |
| 8 | ||
| 9 | Over the years, this website has changed URLs, styles, content focus, and much | |
| 10 | more. However, it seems that I am never done, as I am yet again changing this | |
| 11 | page to fit my current needs and wants. | |
| 12 | ||
| 13 | While this site was already minimal (~12kb), it contained a lot of disorganized | |
| 14 | content and some poorly chosen color schemes. | |
| 15 | ||
| 16 | The recent updates attempt to fix these items while focusing on what I truly | |
| 17 | care about here: the content within each page. | |
| 18 | ||
| 19 | ## Recent Changes | |
| 20 | ||
| 21 | I've made quite a few changes to the style of this website today, both in the | |
| 22 | CSS and in the HTML. | |
| 23 | ||
| 24 | Here are some of the key takeaways from today's changes: | |
| 25 | ||
| 26 | 1. Reduce the width of the website from `60em` to `40em`. | |
| 27 | 2. Remove breadcrumb navigation and replaced with a simple "Return Home" link | |
| 28 | when visiting anything other than the homepage. | |
| 29 | 3. Remove syntax highlighting from code blocks. CSS now loads from a single | |
| 30 | file. | |
| 31 | 4. Move blog posts on the homepage to the top, pushing tertiary content down. | |
| 32 | 5. Update font-family from `monospace` to `sans-serif` for readability. | |
| 33 | ||
| 34 | ## Future Focus | |
| 35 | ||
| 36 | ### Accessibility | |
| 37 | ||
| 38 | My main focus amidst all the changes this past year was accessibility. | |
| 39 | I tried to improve the accessibility of all pages on this site as well as I | |
| 40 | know how. | |
| 41 | For example, I added aria labels to identify different navigation menus in | |
| 42 | November. | |
| 43 | ||
| 44 | I want to continue this in 2023 and learn more about the accessibility features | |
| 45 | I can utilize to help those who may need them. | |
| 46 | ||
| 47 | ### Content First | |
| 48 | ||
| 49 | Beyond accessibility, I am making a concerted effort to bring the content of | |
| 50 | this site to the forefront and push the tertiary information (e.g., About Me, | |
| 51 | Services, etc.) down below the content. | |
| 52 | ||
| 53 | Further, I want to review and edit previous blog posts for grammar, accuracy, | |
| 54 | dead links, and more. Where necessary, I may even add a blurb at the top of old | |
| 55 | posts that are no longer accurate or helpful. | |
| 56 | ||
| 57 | ### Style | |
| 58 | ||
| 59 | As always, I will be searching for ways to reduce distractions and bring a | |
| 60 | minimalistic approach to this site. While there are certainly more drastic | |
| 61 | measures I could take, such as making this a [Shinobi | |
| 62 | Website](https://shinobi.bt.ht) or a [No CSS Website](https://nocss.club), I | |
| 63 | prefer to keep some modern features here. | |
| 64 | ||
| 65 | Two ideas have been floating around in my head, but I am not sure how I feel | |
| 66 | about these approaches yet: | |
| 67 | ||
| 68 | 1. Going all-in with Emacs org-mode and creating a script to auto-publish all my | |
| 69 | blog posts from org-mode to plain HTML files and then letting my CSS style it | |
| 70 | after the fact. | |
| 71 | 2. Focus on publishing for Gemini or Gopher and then utilize a conversion | |
| 72 | program to translate the pages to HTML. | |
| 73 | ||
| 74 | ## Looking Back | |
| 75 | ||
| 76 | As I sit here and finalize the changes, as well as this post, I am happy with | |
| 77 | the current state of this website. It's certainly a long way from my first | |
| 78 | attempts (parallax background images, anyone?) and it's good to see the | |
| 79 | progress. | |
| 80 | ||
| 81 | I can only hope and work hard to ensure that I make more meaningful progress by | |
| 82 | this time next year. | |
blog/2023-01-05-mass-unlike-tumblr-posts.org added +87
| @@ -0,0 +1,87 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-01-05 | |
| 3 | title = "How to Easily Mass Unlike Tumblr Posts With Javascript" | |
| 4 | description = "A quick Javascript snippet that allows you to quickly unlike your Tumblr likes." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## The Dilemma | |
| 8 | ||
| 9 | The dilemma I had was pretty simple: I wanted to unlike all the posts I have | |
| 10 | liked on Tumblr so that I could follow a new focus on blogs and start fresh. | |
| 11 | Otherwise, Tumblr will keep recommending content based on your previous likes. | |
| 12 | ||
| 13 | ## The Solution | |
| 14 | ||
| 15 | I searched the web for a while and noted that most solutions referenced Tumblr | |
| 16 | setting and dashboard pages that no longer exist. Additionally, I did not want | |
| 17 | to install a third party extension to do this, as some suggested. | |
| 18 | ||
| 19 | Luckily, I used Javascript for a while a few years ago and figured it would be | |
| 20 | easy enough to script a solution, as long as Tumblr had a system for the unlike | |
| 21 | buttons. | |
| 22 | ||
| 23 | ### Identifying Unlike Buttons | |
| 24 | ||
| 25 | Tumblr's unlike buttons are structured as you can see in the following code | |
| 26 | block. All unlike buttons have an `aria-label` with a value of `Unlike`. | |
| 27 | ||
| 28 | ```html | |
| 29 | <button class="TRX6J" aria-label="Unlike"> | |
| 30 | <span class="EvhBA B1Z5w ztpfZ" tabindex="-1"> | |
| 31 | <svg xmlns="http://www.w3.org/2000/svg" height="21" width="23" | |
| 32 | role="presentation"> | |
| 33 | <use href="#managed-icon__like-filled"></use> | |
| 34 | </svg> | |
| 35 | </span> | |
| 36 | </button> | |
| 37 | ``` | |
| 38 | ||
| 39 | ### Running a Script to Unlike All Likes | |
| 40 | ||
| 41 | To run this script, you will need to load the [Likes | | |
| 42 | Tumblr](https://www.tumblr.com/likes) page while logged in to your account. | |
| 43 | ||
| 44 | Further, be sure to scroll down to the bottom and force Tumblr to load more | |
| 45 | posts so that this script unlikes more posts at a time. | |
| 46 | ||
| 47 | Once you are logged in and the page is loaded, open the Developer Tools and be | |
| 48 | sure you're on the "Console" tab. It should look something like this (this is | |
| 49 | in Firefox, Chromium should be similar): | |
| 50 | ||
| 51 |  | |
| 53 | ||
| 54 | All you need to do is paste the following snippet into the dev console. This | |
| 55 | code will collect all unlike buttons (`elements`) and then click each button to | |
| 56 | unlike it. | |
| 57 | ||
| 58 | Optionally, you can comment-out the line `elements[i].click();` and uncomment | |
| 59 | the `console.log()` lines to simply print out information without performing any | |
| 60 | actions. This can be useful to debug issues or confirm that the code below isn't | |
| 61 | doing anything you don't want it to. | |
| 62 | ||
| 63 | ```javascript | |
| 64 | const elements = document.querySelectorAll('[aria-label="Unlike"]'); | |
| 65 | // console.log(elements); // 👉 [button] | |
| 66 | ||
| 67 | for (let i=0; i < elements.length; i++) { | |
| 68 | // console.log(elements[i]); | |
| 69 | elements[i].click(); | |
| 70 | } | |
| 71 | ``` | |
| 72 | ||
| 73 | ## Results | |
| 74 | ||
| 75 | The results were quick for my situation, as it unliked ~200 posts within 2-3 | |
| 76 | seconds. I am not sure how this will perform on larger sets of likes (or if | |
| 77 | Tumblr has a limit to unliking posts). | |
| 78 | ||
| 79 | You can see the below screenshot showing that I pasted the snippet into the | |
| 80 | console, pressed Enter, and then the posts are automatically unliked. | |
| 81 | ||
| 82 |  | |
| 84 | ||
| 85 | Thinking about this further, I would bet that this would be fairly simple to | |
| 86 | package into a browser add-on so that users could install the add-on, go to | |
| 87 | their Likes page, and click a button to run the script. Food for thought. | |
blog/2023-01-08-fedora-remove-login-manager.org added +44
| @@ -0,0 +1,44 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-01-08 | |
| 3 | title = "Remove the Login Manager from Fedora i3" | |
| 4 | description = "Quickly remove Fedora i3's login manager to launch your desktop manually." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Fedora i3's Login Manager | |
| 8 | ||
| 9 | Since I use the i3 spin of Fedora Workstation, I don't like to have a login | |
| 10 | manager installed by default. As of the current version of Fedora i3, the | |
| 11 | default login manager is LightDM. | |
| 12 | ||
| 13 | If this is no longer the case, you can search for currently-installed packages | |
| 14 | with the following command and see if you can identify a different login | |
| 15 | manager. | |
| 16 | ||
| 17 | ```sh | |
| 18 | sudo dnf list installed | |
| 19 | ``` | |
| 20 | ||
| 21 | ## Removing the Login Manager | |
| 22 | ||
| 23 | In order to remove the login manager, simply uninstall the package. | |
| 24 | ||
| 25 | ```sh | |
| 26 | sudo dnf remove lightdm | |
| 27 | ``` | |
| 28 | ||
| 29 | ## Launching i3 Manually | |
| 30 | ||
| 31 | In order to launch i3 manually, you need to set up your X session properly. | |
| 32 | To start, create or edit the `~/.xinitrc` file to include the following at | |
| 33 | the bottom. | |
| 34 | ||
| 35 | ```config | |
| 36 | exec i3 | |
| 37 | ``` | |
| 38 | ||
| 39 | Now, whenever you log in to the TTY, you can launch your desktop with the | |
| 40 | following command. | |
| 41 | ||
| 42 | ```sh | |
| 43 | startx | |
| 44 | ``` | |
blog/2023-01-21-running-flatpak-apps-easily.org added +45
| @@ -0,0 +1,45 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-01-21 | |
| 3 | title = "Running Flatpak Apps with Symlinks" | |
| 4 | description = "Run flatpak apps faster and easier with a simple symlink solution." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Running Flatpak Apps Should Be Faster | |
| 8 | ||
| 9 | If you're like me and use Flatpak for those pesky apps that cannot run on your | |
| 10 | system for one reason or another, you likely get annoyed with opening a terminal | |
| 11 | and manually running the Flatpak app with the lengthy `flatpak run ...` command. | |
| 12 | ||
| 13 | In the past, I manually created aliases in my `.zshrc` file for certain apps. | |
| 14 | For example, an alias would look like the example below. | |
| 15 | ||
| 16 | This would allow me to run the command fast within the terminal, but it | |
| 17 | wouldn't allow me to run it in an application launcher. | |
| 18 | ||
| 19 | ```sh | |
| 20 | # ~/.zshrc | |
| 21 | alias librewolf = "flatpak run io.gitlab.librewolf-community" | |
| 22 | ``` | |
| 23 | ||
| 24 | However, I now use a much faster and better method that integrates with the | |
| 25 | tiling WMs I use and their application launchers - `dmenu` and `bemenu`. | |
| 26 | ||
| 27 | ## Creating Symlinks for Flatpak Apps | |
| 28 | ||
| 29 | Let's use the example of Librewolf below. I can install the application like so: | |
| 30 | ||
| 31 | ```sh | |
| 32 | flatpak install flathub io.gitlab.librewolf-community | |
| 33 | ``` | |
| 34 | ||
| 35 | Once installed, I can create a symlink to link the flatpak app to my new symlink | |
| 36 | in a location commonly included in your PATH. In this case, I chose `/usr/bin`. | |
| 37 | You may need to choose a different location if `/usr/bin` isn't in your PATH. | |
| 38 | ||
| 39 | ```sh | |
| 40 | ln -s /var/lib/flatpak/exports/bin/io.gitlab.librewolf-community /usr/bin/librewolf | |
| 41 | ``` | |
| 42 | ||
| 43 | Once complete, you should be able to launch the app using the command name you | |
| 44 | chose above in the symlink (`librewolf`) from a terminal or from your | |
| 45 | application launcher! | |
blog/2023-01-23-random-mullvad-wireguard.org added +114
| @@ -0,0 +1,114 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-01-23 | |
| 3 | title = "Connecting to a Random Mullvad Wireguard Host" | |
| 4 | description = "Read to see how I used a shell script to randomly connect to a Mullvad Wireguard host each time I run the script." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Mullvad Wireguard | |
| 8 | ||
| 9 | If you're using an OS that does not support one of Mullvad's apps, you're | |
| 10 | likely using the Wireguard configuration files instead. | |
| 11 | ||
| 12 | If not, the first step is to visit Mullvad's | |
| 13 | [Wireguard configuration | |
| 14 | files](https://mullvad.net/en/account/#/wireguard-config) page and download a | |
| 15 | ZIP of the configuration files you want to use. | |
| 16 | ||
| 17 | Personally, I downloaded all configuration files across the world and chose my | |
| 18 | connections using the script below. | |
| 19 | ||
| 20 | Once the files are downloaded, unzip them and move them to your preferred | |
| 21 | location: | |
| 22 | ||
| 23 | ```sh | |
| 24 | cd Downloads | |
| 25 | unzip mullvad_wireguard_linux_all_all.zip | |
| 26 | mkdir ~/mullvad && mv ~/Downloads/*.conf ~/mullvad/ | |
| 27 | ``` | |
| 28 | ||
| 29 | ## Creating a Script to Connect to a Random Host | |
| 30 | ||
| 31 | Once you have a folder of Wireguard configuration files from Mullvad, you can | |
| 32 | create a script to randomly connect to any one of the locations. | |
| 33 | ||
| 34 | Start by creating a shell script - mine is called `vpn.sh`. | |
| 35 | ||
| 36 | ```sh | |
| 37 | nano ~/vpn.sh | |
| 38 | ``` | |
| 39 | ||
| 40 | Within this script, you can paste the following info. Note that I specify `us-*` | |
| 41 | in my script, which means that it will only consider US-based VPN locations. You | |
| 42 | can alter this or simply change it `*` to consider all locations. | |
| 43 | ||
| 44 | ```sh | |
| 45 | #!/bin/sh | |
| 46 | ||
| 47 | ls /home/$USER/mullvad/us-* |sort -R |tail -n 1 |while read file; do | |
| 48 | # Replace `doas` with `sudo` if your machine uses `sudo`, | |
| 49 | # or remove `doas` if users don't need to su to run wg-quick | |
| 50 | doas wg-quick up $file; | |
| 51 | printf "\nCreated Mullvad wireguard connection with file: $file"; | |
| 52 | printf "\n\nPrinting new IP info:\n" | |
| 53 | curl https://am.i.mullvad.net/connected | |
| 54 | done | |
| 55 | ``` | |
| 56 | ||
| 57 | Once you've modified the script to your liking, add executable permissions | |
| 58 | and run the script: | |
| 59 | ||
| 60 | ```sh | |
| 61 | chmod +x ~/vpn.sh | |
| 62 | ~/vpn.sh | |
| 63 | ``` | |
| 64 | ||
| 65 | The output should look like the following: | |
| 66 | ||
| 67 | ```txt | |
| 68 | doas (user@host) password: | |
| 69 | ||
| 70 | # ... The script will process all of the iptables and wg commands here | |
| 71 | ||
| 72 | Created Mullvad wireguard connection with file: /home/user/mullvad/us-nyc-wg-210.conf | |
| 73 | ||
| 74 | Printing new IP info: | |
| 75 | You are connected to Mullvad (server country-city-wg-num). Your IP address is 12.345.678.99 | |
| 76 | ``` | |
| 77 | ||
| 78 | That's all there is to it. You can see your new location and IP via the | |
| 79 | `printf` and `curl` commands included in the script. | |
| 80 | ||
| 81 | You can also go to the [Connection Check | | |
| 82 | Mullvad](https://mullvad.net/en/check/) page to see if you are fully connected | |
| 83 | to Mullvad and if any leaks exist. | |
| 84 | ||
| 85 |  | |
| 86 | ||
| 87 | ## Disconnecting from the Wireguard Connection | |
| 88 | ||
| 89 | If you forget which connection you're using, you can execute the following | |
| 90 | command to see where Wireguard is currently connected: | |
| 91 | ||
| 92 | ```sh | |
| 93 | wg show | |
| 94 | ``` | |
| 95 | ||
| 96 | This command will show you the Wireguard interfaces and should output a | |
| 97 | connection like so: `interface: us-lax-wg-104`. | |
| 98 | ||
| 99 | Once you have this, just disconnect using that files' full path: | |
| 100 | ||
| 101 | ```sh | |
| 102 | wg-quick down /home/user/mullvad/us-lax-wg-104.conf | |
| 103 | ``` | |
| 104 | ||
| 105 | I have a TODO item on figuring out how to easily export an environment variable | |
| 106 | that contains the configuration file's full name, so that I can just execute the | |
| 107 | following: | |
| 108 | ||
| 109 | ```sh | |
| 110 | # Ideal situation if I can export the $file variable to the environment | |
| 111 | wg-quick down $file | |
| 112 | ``` | |
| 113 | ||
| 114 | If you have an idea on how to do this, email me! | |
blog/2023-01-28-self-hosting-wger-workout-manager.org added +148
| @@ -0,0 +1,148 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-01-28 | |
| 3 | title = "Self-Hosting Wger Workout Manager" | |
| 4 | description = "A walkthrough on how to host Wger with Docker on Linux." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Wger: The Self-Hosted Workout Manager | |
| 8 | ||
| 9 | [Wger Workout Manager](https://wger.de) is a fitness tracking tool for those who | |
| 10 | enjoy self-hosting their data. You can also register an account on their main | |
| 11 | website if you'd prefer to try without self-hosting. | |
| 12 | ||
| 13 | ### Features | |
| 14 | ||
| 15 | I didn't see a full listing of features anywhere, so I compiled this list of my | |
| 16 | own after installing wger: | |
| 17 | ||
| 18 | #### Dashboard | |
| 19 | ||
| 20 | - Dashboard view of Workout Schedule, Nutrition Plan, Weight Graph, & last 5 | |
| 21 | Weight Logs | |
| 22 | ||
| 23 | #### Training | |
| 24 | ||
| 25 | - Workout Log | |
| 26 | - Workout Schedule | |
| 27 | - Calendar (shows weight logs and Bad/Neutral/Good days) | |
| 28 | - Gallery (shows images you upload) | |
| 29 | - Workout templates | |
| 30 | - Public templates | |
| 31 | - Exercises | |
| 32 | ||
| 33 | #### Nutrition | |
| 34 | ||
| 35 | - Nutrition plans | |
| 36 | - BMI calculator | |
| 37 | - Daily calories calculator | |
| 38 | - Ingredient overview | |
| 39 | ||
| 40 | #### Body Weight | |
| 41 | ||
| 42 | - Weight overview | |
| 43 | ||
| 44 | ### Documentation | |
| 45 | ||
| 46 | In order to self-host wger, I opted to use the Docker version of the | |
| 47 | application. You can read the README within the | |
| 48 | [wger-project/docker](https://github.com/wger-project/docker) project on GitHub | |
| 49 | for information and installation instructions. | |
| 50 | ||
| 51 | ### Installation | |
| 52 | ||
| 53 | To start the installation, I created a folder for wger and started creating the | |
| 54 | three necessary files: | |
| 55 | ||
| 56 | ```sh | |
| 57 | mkdir ~/wger && mkdir ~/wger/config | |
| 58 | touch ~/wger/docker-compose.yml && \ | |
| 59 | touch ~/wger/config/prod.env && \ | |
| 60 | touch ~/wger/config/nginx.conf | |
| 61 | ``` | |
| 62 | ||
| 63 | Once you have the folders and files created, you will need to copy the contents | |
| 64 | of the `docker-compose.yml`, `prod.env`, and `nginx.conf` from the GitHub link | |
| 65 | above. | |
| 66 | ||
| 67 | A few notes to explain the changes I made to the default files: | |
| 68 | ||
| 69 | - I updated the `ALLOW_REGISTRAION` variable in `prod.env` to `False` after I | |
| 70 | created an account via my LAN connection, **before** I connected this app to a | |
| 71 | publicly-available domain. | |
| 72 | - I uncommented and updated `CSRF_TRUSTED_ORIGINS` to be equal to the public | |
| 73 | version of this app: `https://wger.example.com`. | |
| 74 | - I updated the port within `docker-compose.yml`, within the `nginx` block. The | |
| 75 | port I updated this to will be reflected in my nginx configuration file on the | |
| 76 | server (NOT the wger nginx.conf file). | |
| 77 | ||
| 78 | ### Deploy | |
| 79 | ||
| 80 | Once all files are created and modified to your needs, simply start the | |
| 81 | container. | |
| 82 | ||
| 83 | ```sh | |
| 84 | docker-compose up -d | |
| 85 | ``` | |
| 86 | ||
| 87 | You can now visit the website on your LAN by going to `localhost:YOUR_PORT` or | |
| 88 | by the server's IP, if you're not on the same machine that is running the | |
| 89 | container. | |
| 90 | ||
| 91 | If you wish to connect this app to a public domain name, you'll need to point an | |
| 92 | `A` DNS record from the domain to your server's public IP. You'll then need to | |
| 93 | create a configuration file for whichever web server or reverse proxy you're | |
| 94 | using. | |
| 95 | ||
| 96 | Wger's README suggests the following reverse proxy configuration for Nginx: | |
| 97 | ||
| 98 | ```conf | |
| 99 | upstream wger { | |
| 100 | # This port should match the port in the `nginx` block of docker-compose.yml | |
| 101 | # If the container is running on this same machine, replace this with | |
| 102 | # server 127.0.0.1:8080 | |
| 103 | server 123.456.789.0:8080; | |
| 104 | } | |
| 105 | ||
| 106 | server { | |
| 107 | listen 80; | |
| 108 | listen [::]:443 ssl; | |
| 109 | listen 443 ssl; | |
| 110 | ||
| 111 | location / { | |
| 112 | proxy_pass http://wger; | |
| 113 | proxy_set_header Host $http_host; | |
| 114 | proxy_set_header X-Real-IP $remote_addr; | |
| 115 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
| 116 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 117 | proxy_redirect off; | |
| 118 | } | |
| 119 | ||
| 120 | server_name my.domain.example.com; | |
| 121 | ||
| 122 | ssl_certificate /path/to/https/certificate.crt; | |
| 123 | ssl_certificate_key /path/to/https/certificate.key; | |
| 124 | } | |
| 125 | ``` | |
| 126 | ||
| 127 | ## Thoughts on Wger | |
| 128 | ||
| 129 | I'm still playing around with the app itself, but it seems to be a solid | |
| 130 | all-around workout manager, weight log, and food log. | |
| 131 | ||
| 132 | I like that the weight log graph is fluid and updates quickly. You can also | |
| 133 | import or export data in CSV format if you'd like to move your data elsewhere. | |
| 134 | ||
| 135 | The workout manager is slightly odd, as it requires you to enter sets and reps | |
| 136 | for each exercise when you enter it into the plan. Then, when you add a log | |
| 137 | entry for performing a workout, you then add what you actually performed, in | |
| 138 | terms of reps and weight. | |
| 139 | ||
| 140 | I haven't tried the food log yet and I likely will not, at least for a | |
| 141 | while. | |
| 142 | I have no need for a food log or calorie tracker at the moment. | |
| 143 | ||
| 144 | ### Screenshot Example | |
| 145 | ||
| 146 | You can see an example of a dashboard with placeholder data here: | |
| 147 | ||
| 148 |  | |
blog/2023-02-02-exploring-hare.org added +162
| @@ -0,0 +1,162 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-02-02 | |
| 3 | title = "Exploring the Hare Programming Language" | |
| 4 | description = "Walking back through my first project using the Hare programming language." | |
| 5 | aliases = ["/blog/testing-out-hare"] | |
| 6 | +++ | |
| 7 | ||
| 8 | ## A Quick Note | |
| 9 | ||
| 10 | By no means am I a professional developer, so this post will be rather short. | |
| 11 | I won't be going into depth on the specification or anything that technical. | |
| 12 | ||
| 13 | Instead, I will simply be talking about how I (a relatively basic hobbyist | |
| 14 | programmer) have been playing with Hare and what intrigues me about the | |
| 15 | language. | |
| 16 | ||
| 17 | ## Hare | |
| 18 | ||
| 19 | The [Hare](https://harelang.org) programming language is a straightforward | |
| 20 | language that should look familiar if you've ever programmed with C, Rust, | |
| 21 | or other languages that aim to build software at the system-level. | |
| 22 | ||
| 23 | The Hare homepage states the following: | |
| 24 | ||
| 25 | > Hare is a systems programming language designed to be simple, stable, and | |
| 26 | > robust. Hare uses a static type system, manual memory management, and | |
| 27 | > minimal runtime. It is well-suited to writing operating systems, system | |
| 28 | > tools, compilers, networking software, and other low-level, high performance | |
| 29 | > tasks. | |
| 30 | ||
| 31 | I have found this all to be true while playing with it for the first time today. | |
| 32 | In the next few sections, I'm going to walk through my installation and first | |
| 33 | program. | |
| 34 | ||
| 35 | ### Installation | |
| 36 | ||
| 37 | I'm currently running Alpine Linux on my Thinkpad, so the installation was quite | |
| 38 | easy as there is a package for Hare in the `apk` repositories. | |
| 39 | ||
| 40 | ```sh | |
| 41 | doas apk add hare hare-doc | |
| 42 | ``` | |
| 43 | ||
| 44 | However, I was able to install Hare from scratch on Fedora Linux a short while | |
| 45 | ago, which was also very easy to do. If you need further instructions and Hare | |
| 46 | doesn't have a package on your system, take a look at the [Hare | |
| 47 | Installation](https://harelang.org/installation/) page. | |
| 48 | ||
| 49 | ### Creating a Test Project | |
| 50 | ||
| 51 | In order to play with the language, I created | |
| 52 | [hare-test](https://git.sr.ht/~cmc/hare-projects) and will be | |
| 53 | putting any of my Hare-related adventures in here. | |
| 54 | ||
| 55 | > **Update:** I also created a simple Hare program for creating a file from user | |
| 56 | > input: | |
| 57 | > [files.ha](https://git.sr.ht/~cmc/hare-projects/tree/main/item/files/files.ha) | |
| 58 | ||
| 59 | Luckily, Hare doesn't require any complex set-up tools or build environment. | |
| 60 | Once you have Hare installed, you simply need to create a file ending with `.ha` | |
| 61 | and you can run a Hare program. | |
| 62 | ||
| 63 | I created a file called `rgb.ha` in order to test out the random number | |
| 64 | generation and passing parameters between functions. | |
| 65 | ||
| 66 | ```sh | |
| 67 | nano rgb.ha | |
| 68 | ``` | |
| 69 | ||
| 70 | Within this file, I was able to easily import a few of the [standard library | |
| 71 | modules](https://harelang.org/tutorials/stdlib/): `fmt`, `math::random`, and | |
| 72 | `datetime`. | |
| 73 | ||
| 74 | With these modules, I created two functions: | |
| 75 | ||
| 76 | 1. `main`: This function calls the `generate_rgb` function and then prints out | |
| 77 | the returned values. | |
| 78 | 2. `generate_rgb`: This function uses the current Unix epoch time to generate a | |
| 79 | pseudo-random value and uses this value to create three more random values | |
| 80 | between 0 and 255. These three numbers represent a color in RGB format. | |
| 81 | ||
| 82 | > **Note**: Some syntax coloring may look odd, as Zola currently doesn't have a | |
| 83 | > syntax highlighting theme for Hare. Instead, I'm using the C theme, which may | |
| 84 | > not be exactly accurate when coloring the code below. | |
| 85 | ||
| 86 | ```c | |
| 87 | use datetime; | |
| 88 | use fmt; | |
| 89 | use math::random; | |
| 90 | ||
| 91 | export fn main() void = { | |
| 92 | const rgb = generate_rgb(); | |
| 93 | fmt::printfln("RGB: ({}, {}, {})", rgb[0], rgb[1], rgb[2])!; | |
| 94 | }; | |
| 95 | ||
| 96 | fn generate_rgb() []u64 = { | |
| 97 | // Use the current Unix epoch time as the seed value | |
| 98 | let datetime = datetime::epochunix(&datetime::now()); | |
| 99 | ||
| 100 | // Generate initial pseudo-random value | |
| 101 | // You must cast the datetime from int to u64 | |
| 102 | let x = random::init(datetime: u64); | |
| 103 | ||
| 104 | // Generate RGB values between (0, 255) using pseudo-random init value | |
| 105 | let r = random::u64n(&x, 255); | |
| 106 | let g = random::u64n(&x, 255); | |
| 107 | let b = random::u64n(&x, 255); | |
| 108 | ||
| 109 | // Structure data as array and return | |
| 110 | let rgb_array: [3]u64 = [r, g, b]; | |
| 111 | return rgb_array; | |
| 112 | }; | |
| 113 | ``` | |
| 114 | ||
| 115 | ### Running a Program | |
| 116 | ||
| 117 | Once you have a Hare file written and ready to run, you simply need to run it: | |
| 118 | ||
| 119 | ```sh | |
| 120 | hare run file.ha | |
| 121 | ``` | |
| 122 | ||
| 123 | You can also compile the program into an executable: | |
| 124 | ||
| 125 | ```sh | |
| 126 | hare build -o example file.ha | |
| 127 | ./example | |
| 128 | ``` | |
| 129 | ||
| 130 | ### Initial Thoughts | |
| 131 | ||
| 132 | #### Documentation Improvements Would Help | |
| 133 | ||
| 134 | While I was able to piece everything together eventually, the biggest downfall | |
| 135 | right now in Hare's documentation. For such a new project, the documentation is | |
| 136 | in a great spot. However, bare specifications don't help as much as a brief | |
| 137 | examples section would. | |
| 138 | ||
| 139 | For example, it took me a while to figure out what the `u64n` function was | |
| 140 | looking for. I could tell that it took two parameters and the second was my max | |
| 141 | value (255), but couldn't figure out what the first value should be. Eventually, | |
| 142 | I inspected the `random.ha` file in the [Hare source | |
| 143 | code](https://git.sr.ht/~sircmpwn/hare/tree/master/item/math/random/random.ha) | |
| 144 | and found the test suite that helped me discover that it needed an `init()` | |
| 145 | value in the form of `&var`. | |
| 146 | ||
| 147 | #### More Basic Modules | |
| 148 | ||
| 149 | This is another point that comes from Hare being new and awaiting more | |
| 150 | contributions, but there are some basic functions that I would personally enjoy | |
| 151 | seeing in Hare, such as one to convert decimal (base 10) values to hexadecimal | |
| 152 | (base 16). | |
| 153 | ||
| 154 | If I'm feeling comfortable with my math, I may work on the list of functions I | |
| 155 | want and see if any can make it into the Hare source code. | |
| 156 | ||
| 157 | #### Overall Thoughts | |
| 158 | ||
| 159 | Overall, I actually really enjoy Hare. It's not as tedious to get a project up | |
| 160 | and running as Rust, but it's also simpler and more user-friendly than learning | |
| 161 | C. I am going to continue playing with it and see if I can make anything of | |
| 162 | particular value. | |
blog/2023-05-22-burnout.org added +43
| @@ -0,0 +1,43 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-05-22 | |
| 3 | title = "Burnout" | |
| 4 | description = "Dealing with burnout." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## RE: Burnout | |
| 8 | ||
| 9 | I recently read [Drew DeVault's post on burnout](https://drewdevault.com/2023/05/01/2023-05-01-Burnout.html) | |
| 10 | around the same time I was pulling out of a burnout rut myself earlier this month. | |
| 11 | Finally, seeing the light at the end of my burnout tunnel made me want to | |
| 12 | write my first post back on this topic. | |
| 13 | ||
| 14 | ## Busy Seasons on Busy Seasons | |
| 15 | ||
| 16 | My career deals with busy seasons, generally driven by client demand. | |
| 17 | This last year, I dealt with a harsh busy season from Aug to Oct 2022 to | |
| 18 | issue a few SOC reports for the period ending 2022-09-30. | |
| 19 | Immediately following that, I had to pivot and found another busy season | |
| 20 | from Oct to Jan for financial statement audits ending on 2022-12-31. | |
| 21 | Then again, supporting other clients from Jan to Mar 2023, followed by my | |
| 22 | current client workload aiming for SOC reports due on 2023-06-30. | |
| 23 | ||
| 24 | The result? A busy season that has lasted from August 2022 through today. I | |
| 25 | will likely be rushing throughout the next month or two before I have a brief | |
| 26 | break and need to focus on the 2023-09-30 SOC reports again. While auditing and | |
| 27 | consulting always involve a busy season, this is the first time I've | |
| 28 | had one last 9+ months without a break. | |
| 29 | ||
| 30 | While it's been tough, I have a handful of breaks pre-planned throughout this | |
| 31 | next cycle and should be able to moderate the level of commitment required for | |
| 32 | each client. | |
| 33 | ||
| 34 | ## Refocusing | |
| 35 | ||
| 36 | Outside of work, I finally have time to work on hobbies such as this website, | |
| 37 | programming, athletics, games, etc. | |
| 38 | ||
| 39 | You may have noticed my absence if you're in the same channels, forums, and | |
| 40 | rooms that I am, but I should finally be active again. | |
| 41 | ||
| 42 | I'm hoping to break an item out of my backlog soon and start working on building | |
| 43 | a new project or hack around with a stale one. | |
blog/2023-06-08-goaccess-geoip.org added +65
| @@ -0,0 +1,65 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-08T22:05:00Z | |
| 3 | title = "Inspecting Nginx Logs with GoAccess and MaxMind GeoIP Data" | |
| 4 | description = "" | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | [GoAccess](https://goaccess.io/) is an open source real-time web log analyzer | |
| 10 | and interactive viewer that runs in a terminal in *nix systems or through your | |
| 11 | browser. | |
| 12 | ||
| 13 | ## Installation | |
| 14 | ||
| 15 | To start, you'll need to install GoAccess for your OS. Here's an example for | |
| 16 | Debian-based distros: | |
| 17 | ||
| 18 | ```bash | |
| 19 | sudo apt install goaccess | |
| 20 | ``` | |
| 21 | ||
| 22 | Next, find any number of the MaxMind GeoIP database files on GitHub or another | |
| 23 | file hosting website. We're going to use P3TERX's version in this example: | |
| 24 | ||
| 25 | ```bash | |
| 26 | wget https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-City.mmdb | |
| 27 | ``` | |
| 28 | ||
| 29 | Be sure to save this file in an easy to remember location! | |
| 30 | ||
| 31 | ## Usage | |
| 32 | ||
| 33 | In order to utilize the full capabilities of GoAccess and MMDB, start with the | |
| 34 | command template below and customize as necessary. This will export an HTML | |
| 35 | view of the GoAccess dashboard, showing all relevant information related to | |
| 36 | that site's access log. You can also omit the `-o output.html` parameter if you | |
| 37 | prefer to view the data within the CLI instead of creating an HTML file. | |
| 38 | ||
| 39 | With the addition of the GeoIP Database parameter, section `16 - Geo Location` | |
| 40 | will be added with the various countries that are associated with the collected | |
| 41 | IP addresses. | |
| 42 | ||
| 43 | ```bash | |
| 44 | zcat /var/log/nginx/example.access.log.*.gz | goaccess \ | |
| 45 | --geoip-database=/home/user/GeoLite2-City.mmdb \ | |
| 46 | --date-format=%d/%b/%Y \ | |
| 47 | --time-format=%H:%M:%S \ | |
| 48 | --log-format=COMBINED \ | |
| 49 | -o output.html \ | |
| 50 | /var/log/nginx/example.access.log - | |
| 51 | ``` | |
| 52 | ||
| 53 | ### Example Output | |
| 54 | ||
| 55 | See below for an example of the HTML output: | |
| 56 | ||
| 57 |  | |
| 58 | ||
| 59 | You can also see the GeoIP card created by the integration of the MaxMind | |
| 60 | database information. | |
| 61 | ||
| 62 |  | |
| 63 | ||
| 64 | That's all there is to it! | |
| 65 | Informational data is provided in an organized fashion with minimal effort. | |
blog/2023-06-08-self-hosting-baikal-server.org added +157
| @@ -0,0 +1,157 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-08T21:48:00Z | |
| 3 | title = "Self-Hosting Baïkal Server (CalDAV & CardDAV)" | |
| 4 | description = "A quick guide on install Baïkal with docker-compose." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## What is Baïkal? | |
| 8 | ||
| 9 | [Baïkal](https://sabre.io/baikal/) is a lightweight CalDAV + CardDAV server | |
| 10 | that you can self-host on your own machine. | |
| 11 | While I have tried (& failed) to get this CalDAV + CardDAV server running | |
| 12 | before, it was quite easy this time. | |
| 13 | Not really sure what I did differently this time, but I'm documenting my | |
| 14 | process here to ensure I don't forget. | |
| 15 | ||
| 16 | ## Installation | |
| 17 | ||
| 18 | First, create a folder on your server and open a `docker-compose.yml` file for | |
| 19 | editing: | |
| 20 | ||
| 21 | ```bash | |
| 22 | mkdir baikal && cd baikal | |
| 23 | nano docker-compose.yml | |
| 24 | ``` | |
| 25 | ||
| 26 | Within this file, you'll need to paste the information below. You can customize | |
| 27 | the `ports` section to use any port on your server to pass through to port 80 in | |
| 28 | the container. You can also edit the `volumes` section to use docker volumes | |
| 29 | instead of local folders. | |
| 30 | ||
| 31 | ```conf | |
| 32 | version: "2" | |
| 33 | services: | |
| 34 | baikal: | |
| 35 | image: ckulka/baikal:nginx | |
| 36 | restart: always | |
| 37 | ports: | |
| 38 | - "8567:80" | |
| 39 | volumes: | |
| 40 | - ./config:/var/www/baikal/config | |
| 41 | - ./data:/var/www/baikal/Specific | |
| 42 | ``` | |
| 43 | ||
| 44 | Once finished with editing, save and close the file. Then, launch the docker | |
| 45 | container: | |
| 46 | ||
| 47 | ```bash | |
| 48 | sudo docker-compose up -d | |
| 49 | ``` | |
| 50 | ||
| 51 | ## Intial Setup | |
| 52 | ||
| 53 | As long as no issues came up when starting the container, you should be able | |
| 54 | to visit the server's set-up page at `http://<server_ip>:<port>`. | |
| 55 | The application will ask you to create an administrator account and choose | |
| 56 | the database type for your storage. | |
| 57 | Personally, I opted to use SQLite. | |
| 58 | ||
| 59 | Make sure the administrator credentials are adequate to protect against | |
| 60 | common attacks. | |
| 61 | ||
| 62 | ## Creating Users | |
| 63 | ||
| 64 | Once you've set up the application, you will be greeted by the Dashboard page, | |
| 65 | which will show the version of the app, status of the admin/CalDAV/CardDAV | |
| 66 | services, and the number of users, calendars, events, address books, and | |
| 67 | contacts. | |
| 68 | ||
| 69 |  | |
| 70 | ||
| 71 | To create a new user, navigate to the `Users and resources` page. This process | |
| 72 | is as simple as entering a username, password, and email. | |
| 73 | ||
| 74 | Once a user has been created, you can create any number of calendars and address | |
| 75 | books for user, as well as inspect their information. | |
| 76 | ||
| 77 | ## Setting Up a Public URL | |
| 78 | ||
| 79 | Once your application is working locally, you can open access remotely via a URL | |
| 80 | by using a reverse-proxy like Nginx. | |
| 81 | ||
| 82 | As a prerequisite, you must own a domain name and set up DNS records | |
| 83 | to point a domain name to the server hosting Baïkal. | |
| 84 | ||
| 85 | Start by navigating to your web server's configuration directory and create a | |
| 86 | new file for this application. | |
| 87 | ||
| 88 | ```bash | |
| 89 | cd /etc/nginx/sites-available/ | |
| 90 | nano dav | |
| 91 | ``` | |
| 92 | ||
| 93 | Within this file, paste in the configuration from below and change | |
| 94 | `dav.example.com` to match the URL you'll be using. | |
| 95 | ||
| 96 | ```conf | |
| 97 | server { | |
| 98 | if ($host ~ ^[^.]+\.example\.com$) { | |
| 99 | return 301 https://$host$request_uri; | |
| 100 | } | |
| 101 | ||
| 102 | listen [::]:80; | |
| 103 | listen 80; | |
| 104 | server_name dav.example.com; | |
| 105 | return 404; | |
| 106 | } | |
| 107 | ||
| 108 | server { | |
| 109 | listen [::]:443 ssl http2; | |
| 110 | listen 443 ssl http2; | |
| 111 | server_name dav.example.com; | |
| 112 | access_log /var/log/nginx/dav.access.log; | |
| 113 | error_log /var/log/nginx/dav.error.log; | |
| 114 | ||
| 115 | location / { | |
| 116 | proxy_http_version 1.1; | |
| 117 | proxy_pass http://localhost:8567; | |
| 118 | proxy_set_header Host $host; | |
| 119 | } | |
| 120 | ||
| 121 | ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; | |
| 122 | ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; | |
| 123 | include /etc/letsencrypt/options-ssl-nginx.conf; | |
| 124 | ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; | |
| 125 | } | |
| 126 | ``` | |
| 127 | ||
| 128 | For Nginx on Ubuntu, you'll need to symlink the configuration file to the | |
| 129 | `sites-enabled` directory and then restart Nginx. | |
| 130 | ||
| 131 | ```bash | |
| 132 | sudo ln -s /etc/nginx/sites-available/dav /etc/nginx/sites-enabled/dav | |
| 133 | sudo systemctl restart nginx.service | |
| 134 | ``` | |
| 135 | ||
| 136 | At this point, the Baïkal server should be available over the internet at the | |
| 137 | URL configured above! | |
| 138 | ||
| 139 | ## Configuring Clients | |
| 140 | ||
| 141 | Lastly, configuring clients and applications is essential to making sure the DAV | |
| 142 | server is being fully utilized. | |
| 143 | ||
| 144 | You can also use the `Users and resources` page to inspect a user's personal | |
| 145 | link to their calendars and address books by clicking the info (i) button. It | |
| 146 | will show a URI like | |
| 147 | `/dav.php/calendars/your-user/default/`. | |
| 148 | ||
| 149 | However, I found that the following URL works for most applications: | |
| 150 | `/dav.php/principals/your-user/`. | |
| 151 | ||
| 152 | I used the `principals` URL above for Thunderbird (calendar, tasks, and | |
| 153 | contacts), as well as iOS (calendar, tasks, and contacts) and everything works | |
| 154 | flawlessly so far. | |
| 155 | ||
| 156 | Syncing is quick between the server and clients, and I haven't seen any | |
| 157 | disruptions in the service or data integrity. | |
blog/2023-06-18-unifi-ip-blocklist.org added +77
| @@ -0,0 +1,77 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-18 | |
| 3 | title = "Block IP Addresses and Subnets with Unifi Network Firewall" | |
| 4 | description = "A short tutorial on how to create IP Groups in Unifi's Network application and block them via the firewall." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Identifying Abusive IPs | |
| 8 | ||
| 9 | If you're like me and use Unifi network equipment at the edge of the network you | |
| 10 | manage, you may know that Unifi is only somewhat decent at identifying and | |
| 11 | blocking IPs that represent abusive or threat actors. | |
| 12 | ||
| 13 | While Unifi has a [threat | |
| 14 | management](https://help.ui.com/hc/en-us/articles/360006893234-UniFi-Gateway-Threat-Management) | |
| 15 | tool inside their Network application, it can be lacking in functionality and | |
| 16 | identification. For example, I have my UDM Pro set to identify and block almost | |
| 17 | all categories of threats available within the Unifi settings. However, I | |
| 18 | regularly identify abusive actors on my web server via the server logs. | |
| 19 | ||
| 20 | In addition, I have identified IP addresses and subnets directly within Unifi's | |
| 21 | logs that the UDM did not block for whatever reason. | |
| 22 | ||
| 23 | This guide is meant to be another step in the process to manually block abusive | |
| 24 | IP addresses or subnets that you have identified but are not being automatically | |
| 25 | blocked yet. | |
| 26 | ||
| 27 | ## Create an IP Group Profile | |
| 28 | ||
| 29 | To start, login to the Unifi machine's web GUI and navigate to the Network app > | |
| 30 | Settings > Profiles. | |
| 31 | ||
| 32 | Within this page, choose the `IP Groups` tab and click `Create New`. | |
| 33 | ||
| 34 |  | |
| 35 | ||
| 36 | Each IP Group profile can be used as one of three options: | |
| 37 | ||
| 38 | 1. Port Group | |
| 39 | 2. IPv4 Address/Subnet | |
| 40 | 3. IPv6 Address/Subnet | |
| 41 | ||
| 42 | In this example, I'm creating an IPv4 Address/Subnet group and adding a few | |
| 43 | different IP addresses and a subnet. | |
| 44 | Once you've added all IP addresses and subnets, click the `Apply` button | |
| 45 | that should appear at the bottom. | |
| 46 | ||
| 47 |  | |
| 48 | ||
| 49 | At this point, the IPv4 Address/Subnet has been created but not yet used. | |
| 50 | ||
| 51 | ## Drop IP Group Profile via the Unifi Firewall | |
| 52 | ||
| 53 | To instruct the Unifi machine to block the profile we just created, we need | |
| 54 | to navigate to the Network app > Settings > Firewall & Security. | |
| 55 | ||
| 56 | Within this screen, find the Firewall Rules table and click `Create Entry`. This | |
| 57 | entry should contain the following settings: | |
| 58 | ||
| 59 | - Type: `Internet In` | |
| 60 | - Description: `<Your Custom Rule>` | |
| 61 | - Rule Applied: `Before Predefined Rules` | |
| 62 | - Action: `Drop` | |
| 63 | - Source Type: `Port/IP Group` | |
| 64 | - IPv4 Address Group: `<Name of the Group Profile You Created Above>` | |
| 65 | ||
| 66 | Customize the remaining configurations to your liking, and then save and enable | |
| 67 | the firewall rule. | |
| 68 | ||
| 69 |  | |
| 70 | ||
| 71 | Once enabled, the Unifi machine will be able to drop all incoming connections | |
| 72 | from the defined IP addresses and subnets within the created profile. | |
| 73 | ||
| 74 | > As a personal aside to this topic, I'm looking for a convenient way to update | |
| 75 | > the firewall rules or profiles remotely (within the LAN) from the web server | |
| 76 | > to accelerate this process. If you have an idea on how to automatically update | |
| 77 | > Unifi IP groups or firewall rules, let me know! | |
blog/2023-06-20-audit-review-cheatsheet.org added +75
| @@ -0,0 +1,75 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-20 | |
| 3 | title = "Cheatsheet: Review Audit Test Results" | |
| 4 | description = "A handy cheatsheet for reviewing audit testing on FSA and SOC report engagements." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | This post is a *very* brief overview on the basic process to review audit | |
| 10 | test results, focusing on work done as part of a financial statement audit | |
| 11 | (FSA) or service organization controls (SOC) report. | |
| 12 | ||
| 13 | While there are numerous different things to review and look for - all varying | |
| 14 | wildly depending on the report, client, and tester - this list serves as a solid | |
| 15 | base foundation for a reviewer. | |
| 16 | ||
| 17 | I have used this throughout my career as a starting point to my reviews, and it | |
| 18 | has worked wonders for creating a consistent and objective template to my | |
| 19 | reviews. The goal is to keep this base high-level enough to be used on a wide | |
| 20 | variety of engagements, while still ensuring that all key areas are covered. | |
| 21 | ||
| 22 | ## Cheatsheet | |
| 23 | ||
| 24 | 1. [ ] Check all documents for spelling and grammar. | |
| 25 | 2. [ ] Ensure all acronyms are fully explained upon first use. | |
| 26 | 3. [ ] For all people referenced, use their full names and job titles upon | |
| 27 | first use. | |
| 28 | 4. [ ] All supporting documents must cross-reference to the lead sheet and | |
| 29 | vice-versa. | |
| 30 | 5. [ ] Verify that the control has been adequately tested: | |
| 31 | - [ ] **Test of Design**: Did the tester obtain information regarding how | |
| 32 | the control should perform normally and abnormally (e.g., emergency | |
| 33 | scenarios)? | |
| 34 | - [ ] **Test of Operating Effectiveness**: Did the tester inquire, observe, | |
| 35 | inspect, or re-perform sufficient evidence to support their | |
| 36 | conclusion over the control? Inquiry alone is not adequate! | |
| 37 | 6. [ ] For any information used in the control, whether by the control operator | |
| 38 | or by the tester, did the tester appropriately document the source (system or | |
| 39 | person), extraction method, parameters, and completeness and accuracy (C&A)? | |
| 40 | - [ ] For any reports, queries, etc. used in the extraction, did the tester | |
| 41 | include a copy and notate C&A considerations? | |
| 42 | 7. [ ] Did the tester document the specific criteria that the control is being | |
| 43 | tested against? | |
| 44 | 8. [ ] Did the tester notate in the supporting documents where each criterion | |
| 45 | was satisfied? | |
| 46 | 9. [ ] If testing specific policies or procedures, are the documents adequate? | |
| 47 | - [ ] e.g., a test to validate that a review of policy XYZ occurs | |
| 48 | periodically should also evaluate the sufficiency of the policy itself, if | |
| 49 | meant to cover the risk that such a policy does not exist and is not | |
| 50 | reviewed. | |
| 51 | 10. [ ] Does the test cover the appropriate period under review? | |
| 52 | - [ ] If the test is meant to cover only a portion of the audit period, do | |
| 53 | other controls exist to mitigate the risks that exist for the remainder of | |
| 54 | the period? | |
| 55 | 11. [ ] For any computer-aided audit tools (CAATs) or other automation | |
| 56 | techniques used in the test, is the use of such tools explained and | |
| 57 | appropriately documented? | |
| 58 | 12. [ ] If prior-period documentation exists, are there any missing pieces of | |
| 59 | evidence that would further enhance the quality of the test? | |
| 60 | 13. [ ] Was any information discovered during the walkthrough or inquiry phase | |
| 61 | that was not incorporated into the test? | |
| 62 | 14. [ ] Are there new rules or expectations from your company's internal | |
| 63 | guidance or your regulatory bodies that would affect the audit approach for this | |
| 64 | control? | |
| 65 | 15. [ ] Was an exception, finding, or deficiency identified as a result of this | |
| 66 | test? | |
| 67 | - [ ] Was the control deficient in design, operation, or both? | |
| 68 | - [ ] What was the root cause of the finding? | |
| 69 | - [ ] Does the finding indicate other findings or potential fraud? | |
| 70 | - [ ] What's the severity and scope of the finding? | |
| 71 | - [ ] Do other controls exist as a form of compensation against the | |
| 72 | finding's severity, and do they mitigate the risk within the control | |
| 73 | objective? | |
| 74 | - [ ] Does the finding exist at the end of the period, or was it resolved | |
| 75 | within the audit period? | |
| \ No newline at end of file | ||
blog/2023-06-23-byobu.org added +68
| @@ -0,0 +1,68 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-23T19:15:28 | |
| 3 | title = "Byobu" | |
| 4 | description = "A quick look at byobu, a text based window manager." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Byobu | |
| 8 | ||
| 9 | [byobu](https://www.byobu.org) is a command-line tool that allows you to use | |
| 10 | numerous screens within a single terminal emulator instance. More specifically, | |
| 11 | it's a text based window manager, using either `screen` or `tmux`. | |
| 12 | ||
| 13 | This post is mostly just a self-reference as I explore byobu, so I may come back | |
| 14 | later and update this post with more content. | |
| 15 | ||
| 16 | ### Screenshot | |
| 17 | ||
| 18 | Take a look below at my current multi-window set-up in byobu while I write this | |
| 19 | blog post: | |
| 20 | ||
| 21 |  | |
| 22 | ||
| 23 | ## Keybindings | |
| 24 | ||
| 25 | You can open the help menu with either of the following commands; they will both | |
| 26 | open the same manpage: | |
| 27 | ||
| 28 | ```sh | |
| 29 | byobu --help | |
| 30 | # or | |
| 31 | man byobu | |
| 32 | ``` | |
| 33 | ||
| 34 | While the manpage contains a ton of information about the functionality of | |
| 35 | byobu (such as status notifications, sessions, and windows), the first location | |
| 36 | to explore should be the keybindings section. | |
| 37 | ||
| 38 | The keybindings are configured as follows: | |
| 39 | ||
| 40 | ```txt | |
| 41 | byobu keybindings can be user defined in /usr/share/byobu/keybindings/ (or | |
| 42 | within .screenrc if byobu-export was used). The common key bindings are: | |
| 43 | ||
| 44 | F2 - Create a new window | |
| 45 | F3 - Move to previous window | |
| 46 | F4 - Move to next window | |
| 47 | F5 - Reload profile | |
| 48 | F6 - Detach from this session | |
| 49 | F7 - Enter copy/scrollback mode | |
| 50 | F8 - Re-title a window | |
| 51 | F9 - Configuration Menu | |
| 52 | F12 - Lock this terminal | |
| 53 | shift-F2 - Split the screen horizontally | |
| 54 | ctrl-F2 - Split the screen vertically | |
| 55 | shift-F3 - Shift the focus to the previous split region | |
| 56 | shift-F4 - Shift the focus to the next split region | |
| 57 | shift-F5 - Join all splits | |
| 58 | ctrl-F6 - Remove this split | |
| 59 | ctrl-F5 - Reconnect GPG and SSH sockets | |
| 60 | shift-F6 - Detach, but do not logout | |
| 61 | alt-pgup - Enter scrollback mode | |
| 62 | alt-pgdn - Enter scrollback mode | |
| 63 | Ctrl-a $ - show detailed status | |
| 64 | Ctrl-a R - Reload profile | |
| 65 | Ctrl-a ! - Toggle key bindings on and off | |
| 66 | Ctrl-a k - Kill the current window | |
| 67 | Ctrl-a ~ - Save the current window's scrollback buffer | |
| 68 | ``` | |
blog/2023-06-23-self-hosting-convos-irc.org added +164
| @@ -0,0 +1,164 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-23T20:50:00 | |
| 3 | title = "Self-Hosting Convos IRC Web Client" | |
| 4 | description = "" | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Convos | |
| 8 | ||
| 9 | [Convos](https://convos.chat/) is an always-online web client for IRC. It | |
| 10 | has a few features that made it attractive to me as a self-hosted option: | |
| 11 | ||
| 12 | - Extremely simple Docker Compose installation method. | |
| 13 | - Runs in the background and monitors chats even while you're not logged in. | |
| 14 | - Neatly organized sidebar for conversation and client settings. | |
| 15 | - Ability to connect to different hosts and create profiles for hosts. | |
| 16 | - By default, registration is closed to the public. You can enable public | |
| 17 | registration on the Settings page or generate invitation links on the | |
| 18 | Users page. | |
| 19 | - Customization of the client theme, organization name and URL, admin email, | |
| 20 | and video service. | |
| 21 | ||
| 22 | ## Docker Installation | |
| 23 | ||
| 24 | To install Convos, simply create a directory and a `docker-compose.yml` file: | |
| 25 | ||
| 26 | ```sh | |
| 27 | mkdir ~/convos && cd ~/convos | |
| 28 | nano docker-compose.yml | |
| 29 | ``` | |
| 30 | ||
| 31 | With the `docker-compose.yml` file open, paste the configuration below into | |
| 32 | the file. | |
| 33 | You can customize the host port to be something unique, such as `21897:3000`. | |
| 34 | You can also change the `data` folder to be a docker volume instead, if you | |
| 35 | prefer. | |
| 36 | ||
| 37 | ```config | |
| 38 | version: '3' | |
| 39 | ||
| 40 | services: | |
| 41 | convos: | |
| 42 | image: 'convos/convos:stable' | |
| 43 | ports: | |
| 44 | - '3000:3000' | |
| 45 | volumes: | |
| 46 | - './data:/data' | |
| 47 | environment: | |
| 48 | - CONVOS_REVERSE_PROXY=1 | |
| 49 | restart: always | |
| 50 | ``` | |
| 51 | ||
| 52 | Save the `docker-compose.yml` file and bring the container up: | |
| 53 | ||
| 54 | ```sh | |
| 55 | sudo docker-compose up -d | |
| 56 | ``` | |
| 57 | ||
| 58 | At this point, Convos is available at `<server_ip>:3000` but not available | |
| 59 | to the public. | |
| 60 | ||
| 61 | ## Nginx Reverse Proxy | |
| 62 | ||
| 63 | If you're using Nginx, you can create a configuration file for the convos | |
| 64 | application. | |
| 65 | Start by opening the file: | |
| 66 | ||
| 67 | ```sh | |
| 68 | nano /etc/nginx/sites-available/convos | |
| 69 | ``` | |
| 70 | ||
| 71 | Within the nginx configuration file, paste the following content and be sure | |
| 72 | to update `convos.example.com` to match your domain and `127.0.0.1:3000` to | |
| 73 | match the port you opened in the `docker-compose.yml` file. | |
| 74 | ||
| 75 | ```config | |
| 76 | # Host and port where convos is running | |
| 77 | upstream convos_upstream { server 127.0.0.1:3000; } | |
| 78 | ||
| 79 | server { | |
| 80 | if ($host ~ ^[^.]+\.example\.com$) { | |
| 81 | return 301 https://$host$request_uri; | |
| 82 | } | |
| 83 | ||
| 84 | listen [::]:80; | |
| 85 | listen 80; | |
| 86 | server_name convos.example.com; | |
| 87 | return 404; | |
| 88 | } | |
| 89 | ||
| 90 | server { | |
| 91 | listen [::]:443 ssl http2; | |
| 92 | listen 443 ssl http2; | |
| 93 | server_name convos.example.com; | |
| 94 | access_log /var/log/nginx/convos.access.log; | |
| 95 | error_log /var/log/nginx/convos.error.log; | |
| 96 | ||
| 97 | location / { | |
| 98 | proxy_pass http://convos_upstream; | |
| 99 | proxy_http_version 1.1; | |
| 100 | proxy_set_header Upgrade $http_upgrade; | |
| 101 | proxy_set_header Connection "upgrade"; | |
| 102 | client_max_body_size 0; | |
| 103 | proxy_set_header Host $host; | |
| 104 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
| 105 | proxy_set_header X-Request-Base "$scheme://$host/"; | |
| 106 | } | |
| 107 | ||
| 108 | ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; | |
| 109 | ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; | |
| 110 | include /etc/letsencrypt/options-ssl-nginx.conf; | |
| 111 | ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; | |
| 112 | } | |
| 113 | ``` | |
| 114 | ||
| 115 | Once the file is saved, link it to the `sites-enabled` directory and restart | |
| 116 | Nginx. | |
| 117 | ||
| 118 | ```sh | |
| 119 | sudo ln -s /etc/nginx/sites-available/convos /etc/nginx/sites-enabled/convos | |
| 120 | sudo systemctl restart nginx.service | |
| 121 | ``` | |
| 122 | ||
| 123 | ## Screenshots | |
| 124 | ||
| 125 | See the screenshots below for an example of the Convos homepage and an | |
| 126 | excerpt of the chat screen. | |
| 127 | ||
| 128 | There are numerous themes to choose from; the theme shown in the images | |
| 129 | below is Dracula. | |
| 130 | ||
| 131 |  | |
| 133 | ||
| 134 |  | |
| 136 | ||
| 137 | ## Registering a Nickname | |
| 138 | ||
| 139 | Hop into the server channel so that you can message NickServ. In the case of | |
| 140 | Convos, the default sever is libera.chat. | |
| 141 | Simply click the `libera` conversation at the top of the sidebar to open it. | |
| 142 | Once the chat is open, you can claim a nickname by typing: | |
| 143 | ||
| 144 | ```txt | |
| 145 | /nick <nick> | |
| 146 | ``` | |
| 147 | ||
| 148 | If the nickname is available, and you'd like to register the nickname to | |
| 149 | yourself, you'll need to type another command: | |
| 150 | ||
| 151 | ```txt | |
| 152 | /msg NickServ REGISTER | |
| 153 | <password> <email> | |
| 154 | ``` | |
| 155 | ||
| 156 | On libera.chat, the server will send a confirmation email with a command | |
| 157 | that you must message in IRC to verify registration of the nickname: | |
| 158 | ||
| 159 | ```txt | |
| 160 | /msg NickServ VERIFY REGISTER <nick> <verification_code> | |
| 161 | ``` | |
| 162 | ||
| 163 | Once entered, the server should confirm registration of the nickname to the | |
| 164 | supplied email with the password specified. | |
blog/2023-06-28-backblaze-b2.org added +181
| @@ -0,0 +1,181 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-28T16:00:00 | |
| 3 | title = "Getting Started with Backblaze B2 Cloud Storage" | |
| 4 | description = "A simple tutorial to create a Backblaze B2 account for free and back up data from Linux." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | Backblaze [B2 Cloud Storage](https://www.backblaze.com/b2/cloud-storage.html) is | |
| 10 | an inexpensive and reliable on-demand cloud storage and backup solution. | |
| 11 | ||
| 12 | The service starts at $5/TB/month ($0.005/GB/month) with a download rate of | |
| 13 | $0.01/GB/month. | |
| 14 | ||
| 15 | However, there are free tiers: | |
| 16 | ||
| 17 | - The first 10 GB of storage is free. | |
| 18 | - The first 1 GB of data downloaded each day is free. | |
| 19 | - Class A transactions are free. | |
| 20 | - The first 2500 Class B transactions each day are free. | |
| 21 | - The first 2500 Class C transactions each day are free. | |
| 22 | ||
| 23 | You can see which API calls fall into categories A, B, or C here: | |
| 24 | [Pricing Organized by API | |
| 25 | Calls](https://www.backblaze.com/b2/b2-transactions-price.html). | |
| 26 | ||
| 27 | For someone like me, who wants an offsite backup of their server's `/home/` | |
| 28 | directory and various other server configs that fall under 10 GB total, | |
| 29 | Backblaze is a great solution from a financial perspective. | |
| 30 | ||
| 31 | ## Create An Account | |
| 32 | ||
| 33 | To start with Backblaze, you'll need to [create a free | |
| 34 | account](https://www.backblaze.com/b2/sign-up.html) - no payment method is | |
| 35 | required to sign up. | |
| 36 | ||
| 37 | Once you have an account, you can test out the service with their web GUI, their | |
| 38 | mobile app, or their CLI tool. I'm going to use the CLI tool below to test a | |
| 39 | file upload and then sync an entire directory to my Backblaze bucket. | |
| 40 | ||
| 41 | ## Create a Bucket | |
| 42 | ||
| 43 | Before you can start uploading, you need to create a bucket. If you're familiar | |
| 44 | with other object storage services, this will feel familiar. If not, it's pretty | |
| 45 | simple to create one. | |
| 46 | ||
| 47 | As their webpage says: | |
| 48 | ||
| 49 | > A bucket is a container that holds files that are uploaded into B2 Cloud | |
| 50 | > Storage. The bucket name must be globally unique and must have a minimum of 6 | |
| 51 | > characters. A limit of 100 buckets may be created per account. An unlimited | |
| 52 | > number of files may be uploaded into a bucket. | |
| 53 | ||
| 54 | Once you click the `Create a Bucket` button on their webpage or mobile app, you | |
| 55 | need to provide the following: | |
| 56 | ||
| 57 | - Bucket Unique Name | |
| 58 | - Files in Bucket are: `Private` or `Public` | |
| 59 | - Default Encryption: `Disable` or `Enable` | |
| 60 | - Object Lock: `Disable` or `Enable` | |
| 61 | ||
| 62 | For my bucket, I created a private bucket with encryption enabled and object | |
| 63 | lock disabled. | |
| 64 | ||
| 65 | Once your bucket is created, you can test the upload/download feature on their | |
| 66 | web GUI or mobile app! At this point, you have a fully functional bucket and | |
| 67 | account. | |
| 68 | ||
| 69 | ## Linux CLI Tool | |
| 70 | ||
| 71 | ### Installation | |
| 72 | ||
| 73 | To install the `b2` CLI tool, you'll need to download it from the [CLI | |
| 74 | Tools](https://www.backblaze.com/docs/cloud-storage-command-line-tools) page. | |
| 75 | I recommend copying the URL from the link that says `Linux` and using wget to | |
| 76 | download it, as shown below. | |
| 77 | ||
| 78 | Once downloaded, make the file executable and move it to a location on your | |
| 79 | `$PATH`, so that you can execute that command from anywhere on the machine. | |
| 80 | ||
| 81 | ```sh | |
| 82 | wget <b2_cli_url> | |
| 83 | chmod +x b2_linux | |
| 84 | mv b2_linux /usr/bin/b2 | |
| 85 | ``` | |
| 86 | ||
| 87 | ### Log In | |
| 88 | ||
| 89 | The first step after installation is to log in. To do this, execute the | |
| 90 | following command and provide your `<applicationKeyId>` and `<applicationKey>`. | |
| 91 | ||
| 92 | If you don't want to provide these values in the command itself, you can simply | |
| 93 | execute the base command and it will request them in an interactive prompt. | |
| 94 | ||
| 95 | ```sh | |
| 96 | # if you want to provide the keys directly: | |
| 97 | b2 authorize-account [<applicationKeyId>] [<applicationKey>] | |
| 98 | ||
| 99 | # or, if you don't want your keys in your shell history: | |
| 100 | b2 authorize-account | |
| 101 | ``` | |
| 102 | ||
| 103 | ### Upload a Test File | |
| 104 | ||
| 105 | In order to test the functionality of the CLI tool, I'll start by uploading | |
| 106 | a single test file to the bucket I created above. | |
| 107 | We can do this with the `upload_file` function. | |
| 108 | ||
| 109 | The command is issued as follows: | |
| 110 | ||
| 111 | ```sh | |
| 112 | b2 upload_file <bucket_name> <local_file> <remote_file> | |
| 113 | ``` | |
| 114 | ||
| 115 | In my situation, I executed the following command with my username. | |
| 116 | ||
| 117 | ```sh | |
| 118 | b2 upload_file my_unique_bucket /home/<user>/test.md test.md | |
| 119 | ``` | |
| 120 | ||
| 121 | To confirm that the file was uploaded successfully, list the files in your | |
| 122 | bucket: | |
| 123 | ||
| 124 | ```sh | |
| 125 | b2 ls <bucket_name> | |
| 126 | ``` | |
| 127 | ||
| 128 | ```txt | |
| 129 | test.md | |
| 130 | ``` | |
| 131 | ||
| 132 | ### Sync a Directory | |
| 133 | ||
| 134 | If you have numerous files, you can use the `sync` function to perform | |
| 135 | functionality similar to `rsync`, where you can check what's in your bucket | |
| 136 | and sync anything that is new or modified. | |
| 137 | ||
| 138 | The command is issued as follows: | |
| 139 | ||
| 140 | ```sh | |
| 141 | b2 sync <source file location> <B2 bucket destination> | |
| 142 | ``` | |
| 143 | ||
| 144 | In my case, I can sync my user's entire home directory to my bucket without | |
| 145 | specifying any of the files directly: | |
| 146 | ||
| 147 | ```sh | |
| 148 | b2 sync /home/<user>/ "b2://<bucketName>/home/<user>" | |
| 149 | ``` | |
| 150 | ||
| 151 | ## Caveats | |
| 152 | ||
| 153 | ### Timing of Updates to the Web GUI | |
| 154 | ||
| 155 | When performing actions over a bucket, there is a slight delay in the web | |
| 156 | GUI when inspecting a bucket or its file. | |
| 157 | Note that simple actions such as uploading or deleting files may have a | |
| 158 | delay of a few minutes up to 24 hours. | |
| 159 | In my experience (<10 GB and ~20,000 files), any actions took only a few | |
| 160 | minutes to update across clients. | |
| 161 | ||
| 162 | ### Symlinks | |
| 163 | ||
| 164 | Note that symlinks are resolved by b2, so if you have a link from | |
| 165 | `/home/<user>/nas-storage` that symlinks out to a `/mnt/nas-storage` folder | |
| 166 | that has 10TB of data, `b2` will resolve that link and start uploading all 10TB | |
| 167 | of data linked within the folder. | |
| 168 | ||
| 169 | If you're not sure if you have any symlinks, a symlink will look like this | |
| 170 | (note the `->` symbol): | |
| 171 | ||
| 172 | ```sh | |
| 173 | > ls -lha | |
| 174 | lrwxrwxrwx 1 root root 20 Jun 28 13:32 nas -> /mnt/nas-storage/ | |
| 175 | ``` | |
| 176 | ||
| 177 | You can recursively find symlink in a path with the following command: | |
| 178 | ||
| 179 | ```sh | |
| 180 | ls -lR /path/to/search | grep '^l' | |
| 181 | ``` | |
blog/2023-06-30-self-hosting-voyager-lemmy-client.org added +124
| @@ -0,0 +1,124 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-06-30 | |
| 3 | title = "Self Hosting Voyager - A Lemmy Web Client" | |
| 4 | description = "A guide on how to self-host your own instance of Voyager, a mobile-first Lemmy web client." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Installation Guide | |
| 8 | ||
| 9 | [Voyager](https://github.com/aeharding/voyager) is a mobile-first Lemmy web | |
| 10 | client, based on iOS design standards. It follows very closely to Apollo's | |
| 11 | design. | |
| 12 | ||
| 13 | This post is a guide showing how I was able to build and launch my own instance | |
| 14 | of Voyager via Docker Compose. | |
| 15 | ||
| 16 | ### Clone the Repository | |
| 17 | ||
| 18 | Start by cloning the repository and entering it: | |
| 19 | ||
| 20 | ```sh | |
| 21 | git clone https://github.com/aeharding/voyager | |
| 22 | cd voyager | |
| 23 | ``` | |
| 24 | ||
| 25 | ### Build the Image | |
| 26 | ||
| 27 | With this repository, you can build the image yourself without any further | |
| 28 | configuration. | |
| 29 | When complete, it'll give you the image ID for you to run. | |
| 30 | ||
| 31 | ```sh | |
| 32 | sudo docker build . | |
| 33 | # Successfully built 5f00723cb5be | |
| 34 | ``` | |
| 35 | ||
| 36 | With the image ID above, you can run the container and pass the requested port | |
| 37 | `5314` through or use a custom port, if you wish. | |
| 38 | ||
| 39 | You can also set the `CUSTOM_LEMMY_SERVERS` environment variable if you want to | |
| 40 | add to the default suggested login servers. | |
| 41 | This must be set with a comma separated list of suggested servers. | |
| 42 | The first instance in the list will be the default view for logged-out users. | |
| 43 | ||
| 44 | I will be using a `docker-compose.yml` file to run this container, instead of a | |
| 45 | `docker run` command. | |
| 46 | ||
| 47 | ```sh | |
| 48 | nano docker-compose.yml | |
| 49 | ``` | |
| 50 | ||
| 51 | ```conf | |
| 52 | version: "2" | |
| 53 | services: | |
| 54 | voyager: | |
| 55 | image: 5f00723cb5be | |
| 56 | restart: always | |
| 57 | ports: | |
| 58 | - "<custom_port>:5314" | |
| 59 | environment: | |
| 60 | - CUSTOM_LEMMY_SERVERS=lemmy.dbzer0.com,lemmy.world,lemmy.ml,beehaw.org | |
| 61 | ``` | |
| 62 | ||
| 63 | ```sh | |
| 64 | sudo docker-compose up -d | |
| 65 | ``` | |
| 66 | ||
| 67 | The web app will now be available at the following address: | |
| 68 | `<machine_ip>:<custom_port>`. | |
| 69 | If you are running it on your local device, try `localhost:<custom_port>`. | |
| 70 | ||
| 71 | ### Reverse Proxy | |
| 72 | ||
| 73 | If you want to visit this app via an external URL or domain name, you'll need to | |
| 74 | set up a reverse proxy. | |
| 75 | The example below uses Nginx as a reverse proxy. | |
| 76 | ||
| 77 | Simply create the configuration file, paste the contents below, save the file, | |
| 78 | symlink the file, and restart Nginx. | |
| 79 | ||
| 80 | ```sh | |
| 81 | sudo nano /etc/nginx/sites-available/voyager | |
| 82 | ``` | |
| 83 | ||
| 84 | ```conf | |
| 85 | server { | |
| 86 | if ($host ~ ^[^.]+\.example\.com$) { | |
| 87 | return 301 https://$host$request_uri; | |
| 88 | } | |
| 89 | ||
| 90 | listen [::]:80; | |
| 91 | listen 80; | |
| 92 | server_name voyager.example.com; | |
| 93 | return 404; | |
| 94 | } | |
| 95 | ||
| 96 | server { | |
| 97 | listen [::]:443 ssl http2; | |
| 98 | listen 443 ssl http2; | |
| 99 | server_name voyager.example.com; | |
| 100 | access_log /var/log/nginx/voyager.access.log; | |
| 101 | error_log /var/log/nginx/voyager.error.log; | |
| 102 | ||
| 103 | location / { | |
| 104 | proxy_http_version 1.1; | |
| 105 | proxy_pass http://localhost:5314; | |
| 106 | proxy_set_header Host $host; | |
| 107 | } | |
| 108 | ||
| 109 | ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; | |
| 110 | ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; | |
| 111 | include /etc/letsencrypt/options-ssl-nginx.conf; | |
| 112 | ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; | |
| 113 | } | |
| 114 | ``` | |
| 115 | ||
| 116 | ```sh | |
| 117 | sudo ln sudo ln -s /etc/nginx/sites-available/voyager /etc/nginx/sites-enabled/voyager | |
| 118 | sudo systemctl restart nginx.service | |
| 119 | ``` | |
| 120 | ||
| 121 | The site will now be available at the `server_name` you specified above! | |
| 122 | ||
| 123 | You can visit my instance at [voyager.0x4b1d.org](https://voyager.0x4b1d.org) | |
| 124 | for an example. | |
blog/2023-07-12-mullvad-wireguard-lan.org added +142
| @@ -0,0 +1,142 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-07-12T19:31:00 | |
| 3 | title = "Enabling LAN Access in Mullvad Wireguard Conf Files" | |
| 4 | description = "" | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Download Configuration Files from Mullvad | |
| 8 | ||
| 9 | To begin, you'll need [Wireguard configuration files from | |
| 10 | Mullvad](https://mullvad.net/account/wireguard-config). | |
| 11 | You can choose any of the options as you download them. | |
| 12 | For example, I enabled the kill switch, selected all countries, and selected a | |
| 13 | few content filters. | |
| 14 | ||
| 15 | Once downloaded, unzip the files and move them to the Wireguard folder on your | |
| 16 | system. | |
| 17 | ||
| 18 | ```sh | |
| 19 | cd ~/Downloads | |
| 20 | unzip mullvad_wireguard_linux_all_all.zip | |
| 21 | doas mv *.conf /etc/wireguard/ | |
| 22 | ``` | |
| 23 | ||
| 24 | ### Configuration File Layout | |
| 25 | ||
| 26 | The default configuration files will look something like this: | |
| 27 | ||
| 28 | ```conf | |
| 29 | [Interface] | |
| 30 | # Device: <redacted> | |
| 31 | PrivateKey = <redacted> | |
| 32 | Address = <redacted> | |
| 33 | DNS = <redacted> | |
| 34 | PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT && ip6tables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT | |
| 35 | PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT && ip6tables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT | |
| 36 | ||
| 37 | [Peer] | |
| 38 | PublicKey = <redacted> | |
| 39 | AllowedIPs = <redacted> | |
| 40 | Endpoint = <redacted> | |
| 41 | ``` | |
| 42 | ||
| 43 | > Note: If you didn't select the kill switch option, you won't see the `PostUp` | |
| 44 | > and `PreDown` lines. | |
| 45 | > In this case, you'll need to modify the script below to simply append those | |
| 46 | > lines to the `[Interface]` block. | |
| 47 | ||
| 48 | ## Editing the Configuration Files | |
| 49 | ||
| 50 | Once you have the files, you'll need to edit them and replace the `PostUp` and | |
| 51 | `PreDown` lines to enable LAN access. | |
| 52 | ||
| 53 | I recommend that you do this process as root, since you'll need to be able to | |
| 54 | access files in `/etc/wireguard`, which are generally owned by root. | |
| 55 | You can also try using `sudo` or `doas`, but I didn't test that scenario so you | |
| 56 | may need to adjust, as necessary. | |
| 57 | ||
| 58 | ```sh | |
| 59 | su | |
| 60 | ``` | |
| 61 | ||
| 62 | Create the Python file that we'll be using to update the Wireguard configuration | |
| 63 | files. | |
| 64 | ||
| 65 | ```sh | |
| 66 | nano replace.py | |
| 67 | ``` | |
| 68 | ||
| 69 | Within the Python file, copy and paste the logic below. | |
| 70 | This script will open a directory, loop through every configuration file within | |
| 71 | the directory, and replace the `PostUp` and `PreDown` lines with the new | |
| 72 | LAN-enabled iptables commands. | |
| 73 | ||
| 74 | > Note: If your LAN is on a subnet other than `192.168.1.0/24`, you'll need to | |
| 75 | > update the Python script below appropriately. | |
| 76 | ||
| 77 | ```python | |
| 78 | import os | |
| 79 | import fileinput | |
| 80 | ||
| 81 | print("--- starting ---") | |
| 82 | ||
| 83 | dir = "/etc/wireguard/" | |
| 84 | ||
| 85 | for file in os.listdir(dir): | |
| 86 | print(os.path.join(dir, file)) | |
| 87 | for line in fileinput.input(os.path.join(dir, file), inplace=True): | |
| 88 | if "PostUp" in line: | |
| 89 | print("PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL ! -d 192.168.1.0/24 -j REJECT && ip6tables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT") | |
| 90 | elif "PreDown" in line: | |
| 91 | print("PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL ! -d 192.168.1.0/24 -j REJECT && ip6tables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT") | |
| 92 | else: | |
| 93 | print(line, end="") | |
| 94 | ||
| 95 | print("--- done ---") | |
| 96 | ``` | |
| 97 | ||
| 98 | Once you're done, save and close the file. You can now run the Python script | |
| 99 | and watch as each file is updated. | |
| 100 | ||
| 101 | ```sh | |
| 102 | python3 replace.py | |
| 103 | ``` | |
| 104 | ||
| 105 | To confirm it worked, you can `cat` one of the configuration files to inspect | |
| 106 | the new logic and connect to one to test it out. | |
| 107 | ||
| 108 | ```sh | |
| 109 | cat /etc/wireguard/us-chi-wg-001.conf | |
| 110 | ``` | |
| 111 | ||
| 112 | The configuration files should now look like this: | |
| 113 | ||
| 114 | ```conf | |
| 115 | [Interface] | |
| 116 | # Device: <redacted> | |
| 117 | PrivateKey = <redacted> | |
| 118 | Address = <redacted> | |
| 119 | DNS = <redacted> | |
| 120 | PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL ! -d 192.168.1.0/24 -j REJECT && ip6tables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT | |
| 121 | PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL ! -d 192.168.1.0/24 -j REJECT && ip6tables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT | |
| 122 | ||
| 123 | [Peer] | |
| 124 | PublicKey = <redacted> | |
| 125 | AllowedIPs = <redacted> | |
| 126 | Endpoint = <redacted> | |
| 127 | ``` | |
| 128 | ||
| 129 | If you connect to a Wireguard interface, such as `us-chi-wg-001`, you can test | |
| 130 | your SSH functionality and see that it works even while on the VPN. | |
| 131 | ||
| 132 | ```sh | |
| 133 | wg-quick up us-chi-wg-001 | |
| 134 | ssh user@lan-host | |
| 135 | ``` | |
| 136 | ||
| 137 | To confirm your VPN connection, you can curl Mullvad's connection API: | |
| 138 | ||
| 139 | ```sh | |
| 140 | curl https://am.i.mullvad.net/connected | |
| 141 | # You are connected to Mullvad (server us-chi-wg-001). Your IP address is <redacted> | |
| 142 | ``` | |
blog/2023-07-19-plex-transcoder-errors.org added +62
| @@ -0,0 +1,62 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-07-19T17:00:00 | |
| 3 | title = "How to Fix Plex Error: 'Conversion failed. The transcoder failed to start up.'" | |
| 4 | description = "Read to learn how to fix a common Plex error where the transcoder crashes due to SRT subtitle files." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Plex Transcoder Error | |
| 8 | ||
| 9 | Occasionally, you may see an error in your Plex client that references a | |
| 10 | failure with the transcoder conversion process. | |
| 11 | The specific error wording is: | |
| 12 | ||
| 13 | ```txt | |
| 14 | Conversion failed. The transcoder failed to start up. | |
| 15 | ``` | |
| 16 | ||
| 17 | ## Debugging the Cause | |
| 18 | ||
| 19 | In order to get a better look at what is causing the error, I'm going to observe | |
| 20 | the Plex console while the error occurs. | |
| 21 | To do this, open the Plex web client, go to `Settings` > `Manage` > `Console`. | |
| 22 | Now, try to play the title again and watch to see which errors occur. | |
| 23 | ||
| 24 | In my case, you can see the errors below are related to a subtitle file (`.srt`) | |
| 25 | causing the transcoder to crash. | |
| 26 | ||
| 27 | ```txt | |
| 28 | Jul 19, 2023 16:49:34.945 [140184571120440] Error — Couldn't find the file to stream: /movies/Movie Title (2021)/Movie Title (2021).srt | |
| 29 | Jul 19, 2023 16:49:34.947 [140184532732728] Error — [Req#7611/Transcode/42935159-67C1-4192-9336-DDC6F7BC9330] Error configuring transcoder: TPU: Failed to download sub-stream to temporary file | |
| 30 | Jul 19, 2023 16:49:35.225 [140184532732728] Warning — [Req#760d/Transcode] Got a request to stop a transcode session without a valid session GUID. | |
| 31 | Jul 19, 2023 16:49:45.561 [140184532732728] Info — [Req#7648] AutoUpdate: no updates available | |
| 32 | Jul 19, 2023 16:51:23.402 [140184510081848] Info — Library section 1 (Movies) will be updated because of a change in "/movies/Movie Title (2021)/Movie Title (2021).srt" | |
| 33 | ``` | |
| 34 | ||
| 35 | ## Solving the Error | |
| 36 | ||
| 37 | In my case, I simply removed the subtitle file because it was not critical to | |
| 38 | keep. | |
| 39 | You may also avoid this by turning off subtitles if you don't want to delete the | |
| 40 | file. | |
| 41 | ||
| 42 | ```sh | |
| 43 | rm -rf "/movies/Movie Title (2021)/Movie Title (2021).srt" | |
| 44 | ``` | |
| 45 | ||
| 46 | Once the subtitle is removed from the directory or subtitles are turned off, | |
| 47 | try to play the title again. | |
| 48 | At this point, it should play without error. | |
| 49 | If not, reopen or refresh your Plex client and the Plex server: | |
| 50 | ||
| 51 | ```sh | |
| 52 | sudo systemctl restart plexmediaserver.service | |
| 53 | ``` | |
| 54 | ||
| 55 | ## Related Discussion | |
| 56 | ||
| 57 | Looking at [a related Plex forum post](https://forums.plex.tv/t/subtitles-crashing-plex-transcoder-samsung-q80-tv-with-or-without-hardware-transcode/741441/2), | |
| 58 | it seems that `.srt` subtitles are the core issue here. | |
| 59 | However, Plex does not seem to have a resolution that fixes these errors. | |
| 60 | ||
| 61 | Unfortunately, I would suggest converting subtitle formats, burning the | |
| 62 | subtitles into the title, or simply removing subtitles when they cause crashes. | |
blog/2023-08-18-agile-auditing.org added +152
| @@ -0,0 +1,152 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-08-18T17:11:38+00:00 | |
| 3 | title = "Agile Auditing: An Introduction" | |
| 4 | description = "A quick introduction to Agile, Scrum, and Kanban for audit engagement teams." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## What is Agile Auditing? | |
| 8 | ||
| 9 | [Agile](https://en.wikipedia.org/wiki/Agile_software_development), the | |
| 10 | collaborative philosophy behind many software development methods, has been | |
| 11 | picking up steam as a beneficial tool to use in the external and internal | |
| 12 | auditing world. | |
| 13 | ||
| 14 | This blog post will walk through commonly used terms within Agile, Scrum, | |
| 15 | and Kanban in order to translate these terms and roles into audit-specific | |
| 16 | terms. | |
| 17 | ||
| 18 | Whether your team is in charge of a financial statement audit, an | |
| 19 | attestation (SOC 1, SOC 2, etc.), or a unique internal audit, the terms used | |
| 20 | throughout this post should still apply. | |
| 21 | ||
| 22 | ## Agile | |
| 23 | ||
| 24 | To start, I'll take a look at Agile. | |
| 25 | ||
| 26 | > The Agile methodology is a project management approach that involves | |
| 27 | > breaking the project into phases and emphasizes continuous collaboration | |
| 28 | > and improvement. Teams follow a cycle of planning, executing, and evaluating. | |
| 29 | ||
| 30 | While this approach may seem familiar to what audit teams have historically | |
| 31 | done, an audit team must make distinct changes in their mentality and how | |
| 32 | they approach and manage a project. | |
| 33 | ||
| 34 | ### Agile Values | |
| 35 | ||
| 36 | The Agile Manifesto, written in 2001 at a summit in Utah, contain a set of four | |
| 37 | main values that comprise the Agile approach: | |
| 38 | ||
| 39 | 1. Individuals and interactions over processes and tools. | |
| 40 | 2. Working software over comprehensive documentation. | |
| 41 | 3. Customer collaboration over contract negotiation. | |
| 42 | 4. Responding to change over following a plan. | |
| 43 | ||
| 44 | Beyond the four values, [twelve | |
| 45 | principles](https://agilemanifesto.org/principles.html) were also written as | |
| 46 | part of the summit. | |
| 47 | ||
| 48 | In order to relate these values to an audit or attestation engagement, we | |
| 49 | need to shift the focus from software development to the main goal of an | |
| 50 | engagement: completing sufficient audit testing to address to relevant risks | |
| 51 | over the processes and controls at hand. | |
| 52 | ||
| 53 | Audit Examples: | |
| 54 | ||
| 55 | - Engagement teams must value the team members, client contacts, and their | |
| 56 | interactions over the historical processes and tools that have been used. | |
| 57 | - Engagement teams must value a final report that contains sufficient | |
| 58 | audit documentation over excessive documentation or scope creep. | |
| 59 | - Engagement teams must collaborate with the audit clients as much as | |
| 60 | feasible to ensure that both sides are constantly updated with current | |
| 61 | knowledge of the engagement's status and any potential findings, rather | |
| 62 | than waiting for pre-set meetings or the end of the engagement to communicate. | |
| 63 | - Engagement teams must be able to respond to change in an engagement's | |
| 64 | schedule, scope, or environment to ensure that the project is completed in | |
| 65 | a timely manner and that all relevant areas are tested. | |
| 66 | - In terms of an audit department's portfolio, they must be able to | |
| 67 | respond to changes in their company's or client's environment and be | |
| 68 | able to dynamically change their audit plan accordingly. | |
| 69 | ||
| 70 | ## Scrum | |
| 71 | ||
| 72 | The above section discusses the high-level details of the Agile philosophy | |
| 73 | and how an audit team can potentially mold that mindset into the audit world, | |
| 74 | but how does a team implement these ideas? | |
| 75 | ||
| 76 | There are many methods that use an Agile mindset, but I prefer | |
| 77 | [Scrum](https://en.wikipedia.org/wiki/Scrum_(software_development)). Scrum | |
| 78 | is a framework based on Agile that enables a team to work through a project | |
| 79 | through a series of roles, ceremonies, artifacts, and values. | |
| 80 | ||
| 81 | Let's dive into each of these individually. | |
| 82 | ||
| 83 | ### Scrum Team | |
| 84 | ||
| 85 | A scrum project is only as good as the team running the project. Standard | |
| 86 | scrum teams are separated into three distinct areas: | |
| 87 | ||
| 88 | 1. **Product Owner (Client Contact)**: The client contact is the audit | |
| 89 | equivalent of the product owner in Scrum. They are responsible for | |
| 90 | partnering with the engagement or audit team to ensure progress is being | |
| 91 | made, priorities are established, and clear guidance is given when | |
| 92 | questions or findings arise within each sprint. | |
| 93 | 2. **Scrum Master (Engagement Lead)**: The engagement or audit team lead is | |
| 94 | responsible for coaching the team and the client contact on the scrum | |
| 95 | process, tracking team progress against plan, scheduling necessary | |
| 96 | resources, and helping remove obstacles. | |
| 97 | 3. **Scrum Developers (Engagement Members)**: The engagement or audit team | |
| 98 | is the set of team members responsible for getting the work done. These | |
| 99 | team members will work on each task, report progress, resolve obstacles, | |
| 100 | and collaborate with other team members and the client contact to ensure | |
| 101 | goals are being met. | |
| 102 | ||
| 103 | ### Scrum Ceremonies | |
| 104 | ||
| 105 | Scrum ceremonies are events that are performed on a regular basis. | |
| 106 | ||
| 107 | 1. **Sprint Planning**: The team works together to plan the upcoming sprint | |
| 108 | goal | |
| 109 | and which user stories (tasks) will be added to the sprint to achieve | |
| 110 | that goal. | |
| 111 | 2. **Sprint**: The time period, typically at least one week and no more than one | |
| 112 | month in length, where the team works on the stories and anything in the | |
| 113 | backlog. | |
| 114 | 3. **Daily Scrum**: A very short meeting held each day, typically 15 minutes, to | |
| 115 | quickly emphasize alignment on the sprint goal and plan the next 24 hours. | |
| 116 | Each team member may share what they did the day before, what they'll do | |
| 117 | today, and any obstacles to their work. | |
| 118 | 4. **Sprint Review**: At the end of each sprint, the team will gather and | |
| 119 | discuss the progress, obstacles, and backlog from the previous sprint. | |
| 120 | 5. **Sprint Retrospective**: More specific than the sprint review, the | |
| 121 | retrospective is meant to discuss what worked and what did not work | |
| 122 | during the sprint. This may be processes, tools, people, or even things | |
| 123 | related to the Scrum ceremonies. | |
| 124 | ||
| 125 | One additional ceremony that may be applicable is organizing the backlog. | |
| 126 | This is typically the responsibility of the engagement leader and is meant | |
| 127 | to prioritize and clarify what needs to be done to complete items in the | |
| 128 | backlog. | |
| 129 | ||
| 130 | ### Artifacts | |
| 131 | ||
| 132 | While artifacts are generally not customizable in the audit world (i.e., | |
| 133 | each control test must include some kind of working paper with evidence | |
| 134 | supporting the test results), I wanted to include some quick notes on | |
| 135 | associating scrum artifact terms with an audit. | |
| 136 | ||
| 137 | 1. **Product Backlog**: This is the overall backlog of unfinished audit | |
| 138 | tasks from all prior sprints. | |
| 139 | 2. **Sprint Backlog**: This is the backlog of unfinished audit tasks from | |
| 140 | one individual sprint. | |
| 141 | 3. **Increment**: This is the output of each sprint - generally this is best | |
| 142 | thought of as any documentation prepared during the sprint, such as risk | |
| 143 | assessments, control working papers, deficiency analysis, etc. | |
| 144 | ||
| 145 | ## Kanban | |
| 146 | ||
| 147 | Last but not least, Kanban is a methodology that relies on boards to | |
| 148 | categorize work into distinct, descriptive categories that allow an agile or | |
| 149 | scrum team to effectively plan the work of a sprint or project. | |
| 150 | ||
| 151 | See Atlassian's [Kanban](https://www.atlassian.com/agile/kanban) page for | |
| 152 | more information. | |
blog/2023-09-15-gitweb.org added +74
| @@ -0,0 +1,74 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-09-16T03:10:37+00:00 | |
| 3 | title = "GitWeb via Nginx" | |
| 4 | description = "A short guide on how to serve GitWeb via Nginx." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | [GitWeb](https://git-scm.com/book/en/v2/Git-on-the-Server-GitWeb) is a simple | |
| 10 | web-based visualizer for git repositories. | |
| 11 | By default, GitWeb will only run with the `lighttpd` or `webrick` web servers. | |
| 12 | ||
| 13 | However, this guide will show you how to keep GitWeb running in the background | |
| 14 | and display information for all repositories in a chosen directory. | |
| 15 | ||
| 16 | See below for the final result: | |
| 17 | ||
| 18 |  | |
| 19 | ||
| 20 | ## Install Dependencies | |
| 21 | ||
| 22 | To start, you'll need install the following packages: | |
| 23 | ||
| 24 | ```sh | |
| 25 | sudo apt install git gitweb fcgiwrap nginx | |
| 26 | ``` | |
| 27 | ||
| 28 | ## Configure Nginx | |
| 29 | ||
| 30 | Once installed, create an Nginx configuration file. | |
| 31 | ||
| 32 | ```sh | |
| 33 | sudo nano /etc/nginx/sites-available/git.example.com | |
| 34 | ``` | |
| 35 | ||
| 36 | ```conf | |
| 37 | server { | |
| 38 | listen 80; | |
| 39 | server_name example.com; | |
| 40 | ||
| 41 | location /index.cgi { | |
| 42 | root /usr/share/gitweb/; | |
| 43 | include fastcgi_params; | |
| 44 | gzip off; | |
| 45 | fastcgi_param SCRIPT_NAME $uri; | |
| 46 | fastcgi_param GITWEB_CONFIG /etc/gitweb.conf; | |
| 47 | fastcgi_pass unix:/var/run/fcgiwrap.socket; | |
| 48 | } | |
| 49 | ||
| 50 | location / { | |
| 51 | root /usr/share/gitweb/; | |
| 52 | index index.cgi; | |
| 53 | } | |
| 54 | } | |
| 55 | ``` | |
| 56 | ||
| 57 | To make the configuration active, you need to symlink it and then restart Nginx. | |
| 58 | ||
| 59 | ```sh | |
| 60 | sudo ln -s /etc/nginx/sites-available/git.example.com /etc/nginx/sites-enabled/git.example.com | |
| 61 | sudo systemctl restart nginx.service | |
| 62 | ``` | |
| 63 | ||
| 64 | The GitWeb application should now be available via the URL you set in the Nginx | |
| 65 | configuration above. | |
| 66 | ||
| 67 | ## Customize GitWeb | |
| 68 | ||
| 69 | If you need to, you can customize many things about Gitweb by editing the | |
| 70 | [gitweb.conf](https://git-scm.com/docs/gitweb.conf) file. | |
| 71 | ||
| 72 | ```sh | |
| 73 | sudo nano /etc/gitweb.conf | |
| 74 | ``` | |
blog/2023-09-19-audit-sql-scripts.org added +266
| @@ -0,0 +1,266 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-09-20T01:12:51+00:00 | |
| 3 | title = "Useful SQL Scripts for Auditing Logical Access" | |
| 4 | description = "A handful of SQL scripts in different dialects to gather user permissions on a database." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | When you have to scope a database into your engagement, you may be curious | |
| 10 | how to best extract the information from the database. While there are | |
| 11 | numerous different methods to extract this type of information, I'm going to | |
| 12 | show an example of how to gather all users and privileges from three main | |
| 13 | database types: Oracle, Microsoft SQL, and MySQL. | |
| 14 | ||
| 15 | ## Oracle | |
| 16 | ||
| 17 | You can use the following SQL script to see all users and their privileges | |
| 18 | in an Oracle database: | |
| 19 | ||
| 20 | ```sql | |
| 21 | SELECT | |
| 22 | grantee AS "User", | |
| 23 | privilege AS "Privilege" | |
| 24 | FROM | |
| 25 | dba_sys_privs | |
| 26 | WHERE | |
| 27 | grantee IN (SELECT DISTINCT grantee FROM dba_sys_privs) | |
| 28 | UNION ALL | |
| 29 | SELECT | |
| 30 | grantee AS "User", | |
| 31 | privilege AS "Privilege" | |
| 32 | FROM | |
| 33 | dba_tab_privs | |
| 34 | WHERE | |
| 35 | grantee IN (SELECT DISTINCT grantee FROM dba_tab_privs); | |
| 36 | ``` | |
| 37 | ||
| 38 | This script queries the `dba_sys_privs` and `dba_tab_privs` views to | |
| 39 | retrieve system and table-level privileges respectively. It then combines | |
| 40 | the results using `UNION ALL` to show all users and their associated | |
| 41 | privileges. Please note that this method does not extract information from | |
| 42 | the `dba_role_privs` table - use the method below for that data. | |
| 43 | ||
| 44 | Please note that you might need appropriate privileges (e.g., DBA privileges) | |
| 45 | to access these views, and you should exercise caution when querying system | |
| 46 | tables in a production Oracle database. | |
| 47 | ||
| 48 | ### Alternative Oracle Query | |
| 49 | ||
| 50 | You can also extract each table's information separately and perform | |
| 51 | processing outside the database to explore and determine the information | |
| 52 | necessary for the audit: | |
| 53 | ||
| 54 | ```sql | |
| 55 | SELECT * FROM sys.dba_role_privs; | |
| 56 | SELECT * FROM sys.dba_sys_privs; | |
| 57 | SELECT * FROM sys.dba_tab_privs; | |
| 58 | SELECT * FROM sys.dba_users; | |
| 59 | ``` | |
| 60 | ||
| 61 | ## Microsoft SQL | |
| 62 | ||
| 63 | You can use the following SQL script to see all users and their privileges | |
| 64 | in a Microsoft SQL Server database | |
| 65 | ([source](https://stackoverflow.com/a/30040784)): | |
| 66 | ||
| 67 | ```sql | |
| 68 | /* | |
| 69 | Security Audit Report | |
| 70 | 1) List all access provisioned to a sql user or windows user/group directly | |
| 71 | 2) List all access provisioned to a sql user or windows user/group through a database or application role | |
| 72 | 3) List all access provisioned to the public role | |
| 73 | ||
| 74 | Columns Returned: | |
| 75 | UserName : SQL or Windows/Active Directory user account. This could also be an Active Directory group. | |
| 76 | UserType : Value will be either 'SQL User' or 'Windows User'. This reflects the type of user defined for the | |
| 77 | SQL Server user account. | |
| 78 | DatabaseUserName: Name of the associated user as defined in the database user account. The database user may not be the | |
| 79 | same as the server user. | |
| 80 | Role : The role name. This will be null if the associated permissions to the object are defined at directly | |
| 81 | on the user account, otherwise this will be the name of the role that the user is a member of. | |
| 82 | PermissionType : Type of permissions the user/role has on an object. Examples could include CONNECT, EXECUTE, SELECT | |
| 83 | DELETE, INSERT, ALTER, CONTROL, TAKE OWNERSHIP, VIEW DEFINITION, etc. | |
| 84 | This value may not be populated for all roles. Some built in roles have implicit permission | |
| 85 | definitions. | |
| 86 | PermissionState : Reflects the state of the permission type, examples could include GRANT, DENY, etc. | |
| 87 | This value may not be populated for all roles. Some built in roles have implicit permission | |
| 88 | definitions. | |
| 89 | ObjectType : Type of object the user/role is assigned permissions on. Examples could include USER_TABLE, | |
| 90 | SQL_SCALAR_FUNCTION, SQL_INLINE_TABLE_VALUED_FUNCTION, SQL_STORED_PROCEDURE, VIEW, etc. | |
| 91 | This value may not be populated for all roles. Some built in roles have implicit permission | |
| 92 | definitions. | |
| 93 | ObjectName : Name of the object that the user/role is assigned permissions on. | |
| 94 | This value may not be populated for all roles. Some built in roles have implicit permission | |
| 95 | definitions. | |
| 96 | ColumnName : Name of the column of the object that the user/role is assigned permissions on. This value | |
| 97 | is only populated if the object is a table, view or a table value function. | |
| 98 | */ | |
| 99 | ||
| 100 | --List all access provisioned to a sql user or windows user/group directly | |
| 101 | SELECT | |
| 102 | [UserName] = CASE princ.[type] | |
| 103 | WHEN 'S' THEN princ.[name] | |
| 104 | WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI | |
| 105 | END, | |
| 106 | [UserType] = CASE princ.[type] | |
| 107 | WHEN 'S' THEN 'SQL User' | |
| 108 | WHEN 'U' THEN 'Windows User' | |
| 109 | END, | |
| 110 | [DatabaseUserName] = princ.[name], | |
| 111 | [Role] = null, | |
| 112 | [PermissionType] = perm.[permission_name], | |
| 113 | [PermissionState] = perm.[state_desc], | |
| 114 | [ObjectType] = obj.type_desc,--perm.[class_desc], | |
| 115 | [ObjectName] = OBJECT_NAME(perm.major_id), | |
| 116 | [ColumnName] = col.[name] | |
| 117 | FROM | |
| 118 | --database user | |
| 119 | sys.database_principals princ | |
| 120 | LEFT JOIN | |
| 121 | --Login accounts | |
| 122 | sys.login_token ulogin on princ.[sid] = ulogin.[sid] | |
| 123 | LEFT JOIN | |
| 124 | --Permissions | |
| 125 | sys.database_permissions perm ON perm.[grantee_principal_id] = princ.[principal_id] | |
| 126 | LEFT JOIN | |
| 127 | --Table columns | |
| 128 | sys.columns col ON col.[object_id] = perm.major_id | |
| 129 | AND col.[column_id] = perm.[minor_id] | |
| 130 | LEFT JOIN | |
| 131 | sys.objects obj ON perm.[major_id] = obj.[object_id] | |
| 132 | WHERE | |
| 133 | princ.[type] in ('S','U') | |
| 134 | UNION | |
| 135 | --List all access provisioned to a sql user or windows user/group through a database or application role | |
| 136 | SELECT | |
| 137 | [UserName] = CASE memberprinc.[type] | |
| 138 | WHEN 'S' THEN memberprinc.[name] | |
| 139 | WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI | |
| 140 | END, | |
| 141 | [UserType] = CASE memberprinc.[type] | |
| 142 | WHEN 'S' THEN 'SQL User' | |
| 143 | WHEN 'U' THEN 'Windows User' | |
| 144 | END, | |
| 145 | [DatabaseUserName] = memberprinc.[name], | |
| 146 | [Role] = roleprinc.[name], | |
| 147 | [PermissionType] = perm.[permission_name], | |
| 148 | [PermissionState] = perm.[state_desc], | |
| 149 | [ObjectType] = obj.type_desc,--perm.[class_desc], | |
| 150 | [ObjectName] = OBJECT_NAME(perm.major_id), | |
| 151 | [ColumnName] = col.[name] | |
| 152 | FROM | |
| 153 | --Role/member associations | |
| 154 | sys.database_role_members members | |
| 155 | JOIN | |
| 156 | --Roles | |
| 157 | sys.database_principals roleprinc ON roleprinc.[principal_id] = members.[role_principal_id] | |
| 158 | JOIN | |
| 159 | --Role members (database users) | |
| 160 | sys.database_principals memberprinc ON memberprinc.[principal_id] = members.[member_principal_id] | |
| 161 | LEFT JOIN | |
| 162 | --Login accounts | |
| 163 | sys.login_token ulogin on memberprinc.[sid] = ulogin.[sid] | |
| 164 | LEFT JOIN | |
| 165 | --Permissions | |
| 166 | sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id] | |
| 167 | LEFT JOIN | |
| 168 | --Table columns | |
| 169 | sys.columns col on col.[object_id] = perm.major_id | |
| 170 | AND col.[column_id] = perm.[minor_id] | |
| 171 | LEFT JOIN | |
| 172 | sys.objects obj ON perm.[major_id] = obj.[object_id] | |
| 173 | UNION | |
| 174 | --List all access provisioned to the public role, which everyone gets by default | |
| 175 | SELECT | |
| 176 | [UserName] = '{All Users}', | |
| 177 | [UserType] = '{All Users}', | |
| 178 | [DatabaseUserName] = '{All Users}', | |
| 179 | [Role] = roleprinc.[name], | |
| 180 | [PermissionType] = perm.[permission_name], | |
| 181 | [PermissionState] = perm.[state_desc], | |
| 182 | [ObjectType] = obj.type_desc,--perm.[class_desc], | |
| 183 | [ObjectName] = OBJECT_NAME(perm.major_id), | |
| 184 | [ColumnName] = col.[name] | |
| 185 | FROM | |
| 186 | --Roles | |
| 187 | sys.database_principals roleprinc | |
| 188 | LEFT JOIN | |
| 189 | --Role permissions | |
| 190 | sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id] | |
| 191 | LEFT JOIN | |
| 192 | --Table columns | |
| 193 | sys.columns col on col.[object_id] = perm.major_id | |
| 194 | AND col.[column_id] = perm.[minor_id] | |
| 195 | JOIN | |
| 196 | --All objects | |
| 197 | sys.objects obj ON obj.[object_id] = perm.[major_id] | |
| 198 | WHERE | |
| 199 | --Only roles | |
| 200 | roleprinc.[type] = 'R' AND | |
| 201 | --Only public role | |
| 202 | roleprinc.[name] = 'public' AND | |
| 203 | --Only objects of ours, not the MS objects | |
| 204 | obj.is_ms_shipped = 0 | |
| 205 | ORDER BY | |
| 206 | princ.[Name], | |
| 207 | OBJECT_NAME(perm.major_id), | |
| 208 | col.[name], | |
| 209 | perm.[permission_name], | |
| 210 | perm.[state_desc], | |
| 211 | obj.type_desc--perm.[class_desc] | |
| 212 | ``` | |
| 213 | ||
| 214 | ## MySQL | |
| 215 | ||
| 216 | You can use the following SQL script to see all users and their privileges | |
| 217 | in a MySQL database: | |
| 218 | ||
| 219 | ```shell | |
| 220 | mysql -u root -p | |
| 221 | ``` | |
| 222 | ||
| 223 | Find all users and hosts with access to the database: | |
| 224 | ||
| 225 | ```sql | |
| 226 | SELECT * FROM information_schema.user_privileges; | |
| 227 | ``` | |
| 228 | ||
| 229 | This script retrieves user information and their associated database-level | |
| 230 | privileges from the `information_schema.user_privileges` table in MySQL. It | |
| 231 | lists various privileges such as SELECT, INSERT, UPDATE, DELETE, CREATE, and | |
| 232 | more for each user and database combination. | |
| 233 | ||
| 234 | Please note that you may need appropriate privileges (e.g., `SELECT` | |
| 235 | privileges on `information_schema.user_privileges`) to access this | |
| 236 | information in a MySQL database. Additionally, some privileges like GRANT | |
| 237 | OPTION, EXECUTE, EVENT, and TRIGGER may not be relevant for all users and | |
| 238 | databases. | |
| 239 | ||
| 240 | ### Alternative MySQL Query | |
| 241 | ||
| 242 | You can also grab individual sets of data from MySQL if you prefer to join them | |
| 243 | after extraction. I have marked the queries below with `SELECT ...` and excluded | |
| 244 | most `WHERE` clauses for brevity. You should determine the relevant privileges | |
| 245 | in-scope and query for those privileges to reduce the length of time to query. | |
| 246 | ||
| 247 | ```sql | |
| 248 | -- Global Permissions | |
| 249 | SELECT ... FROM mysql.user; | |
| 250 | ||
| 251 | -- Database Permissions | |
| 252 | SELECT ... FROM mysql.db | |
| 253 | WHERE db = @db_name; | |
| 254 | ||
| 255 | -- Table Permissions | |
| 256 | SELECT ... FROM mysql.tables | |
| 257 | WHERE db = @db_name; | |
| 258 | ||
| 259 | -- Column Permissions | |
| 260 | SELECT ... FROM mysql.columns_priv | |
| 261 | WHERE db = @db_name; | |
| 262 | ||
| 263 | -- Password Configuration | |
| 264 | SHOW GLOBAL VARIABLES LIKE 'validate_password%'; | |
| 265 | SHOW VARIABLES LIKE 'validate_password%'; | |
| 266 | ``` | |
blog/2023-10-04-digital-minimalism.org added +98
| @@ -0,0 +1,98 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-10-05T02:10:47+00:00 | |
| 3 | title = "Digital Minimalism" | |
| 4 | description = "Some personal reflections on digital minimalism." | |
| 5 | +++ | |
| 6 | ||
| 7 | I've written [a note about minimalism](/notes/minimalism) before, but I wanted | |
| 8 | to dedicate some time to reflect on digital minimalism and how I've been able | |
| 9 | to minimize the impact of digital devices in my life. | |
| 10 | ||
| 11 | > These changes crept up on us and happened fast, before we had a chance to step | |
| 12 | > back and ask what we really wanted out of the rapid advances of the past | |
| 13 | > decade. We added new technologies to the periphery of our experience for minor | |
| 14 | > reasons, then woke one morning to discover that they had colonized the core | |
| 15 | > of our daily life. We didn’t, in other words, sign up for the digital world in | |
| 16 | > which we’re currently entrenched; we seem to have stumbled backward into it. | |
| 17 | > *(Digital Minimalism, 2019)* | |
| 18 | ||
| 19 | ## The Principles of Digital Minimalism | |
| 20 | ||
| 21 | As noted in Cal Newport's book, *Digital Minimalism*, there are three main | |
| 22 | principles to digital minimalism that I tend to agree with: | |
| 23 | ||
| 24 | 1. Clutter is costly. | |
| 25 | - Digital minimalists recognize that cluttering their time and attention | |
| 26 | with too many devices, apps, and services creates an overall negative | |
| 27 | cost that can swamp the small benefits that each individual item | |
| 28 | provides in isolation. | |
| 29 | 2. Optimization is important. | |
| 30 | - Digital minimalists believe that deciding a particular technology | |
| 31 | supports something they value is only the first step. To truly extract | |
| 32 | its full potential benefit, it’s necessary to think carefully about how | |
| 33 | they’ll use the technology. | |
| 34 | 3. Intentionality is satisfying. | |
| 35 | - Digital minimalists derive significant satisfaction from their general | |
| 36 | commitment to being more intentional about how they engage with new | |
| 37 | technologies. This source of satisfaction is independent of the specific | |
| 38 | decisions they make and is one of the biggest reasons that minimalism | |
| 39 | tends to be immensely meaningful to its practitioners. | |
| 40 | ||
| 41 | ## Taking Action | |
| 42 | ||
| 43 | In order to put the logic into practice, I've created a few new habits and | |
| 44 | continued performing old habits that are working well: | |
| 45 | ||
| 46 | ### Using Devices With Intention | |
| 47 | ||
| 48 | - I already rarely use "social media", mostly limited to forums such as Hacker | |
| 49 | News and Tildes, so I've just tweaked my behavior to stop looking for content in | |
| 50 | those places when I'm bored. | |
| 51 | - Use devices with intention. Each time I pick up a digital device, there should | |
| 52 | be an intention to use the device to improve my current situation. No more | |
| 53 | endless scrolling or searching for something to interest me. | |
| 54 | ||
| 55 | ### Prevent Distractions | |
| 56 | ||
| 57 | - Disable (most) notifications on all devices. I spent 15-30 minutes going | |
| 58 | through the notifications on my phone, watch, and computer to ensure that only a | |
| 59 | select few apps have the ability to interrupt me: Calendar, Messages, Phone, | |
| 60 | Reminders, & Signal. | |
| 61 | - Disable badges for any apps except the ones mentioned in the bullet above. | |
| 62 | - Set-up focus profiles across devices so that I can enable different modes, | |
| 63 | such as Personal when I only want to see notifications from people I care about | |
| 64 | or Do Not Disturb, where absolutely nothing can interrupt me. | |
| 65 | - Clean up my home screens. This one was quite easy as I already maintain a | |
| 66 | minimalist set-up, but I went extreme by limiting my phone to just eight apps on | |
| 67 | the home screen and four in the dock. If I need another app, I'll have to search | |
| 68 | or use the app library. | |
| 69 | - Remove the work profile from my phone. This was a tough decision as having my | |
| 70 | work profile on my device definitely makes my life easier at times, but it also | |
| 71 | has quite a negative effect when I'm "always online" and can see the | |
| 72 | notifications and team activity 24/7. I believe creating a distinct barrier | |
| 73 | between my work and personal devices will be beneficial in the end. | |
| 74 | ||
| 75 | ### Creating Alternative Activities | |
| 76 | ||
| 77 | This is the most difficult piece, as most of my hobbies and interests lie in the | |
| 78 | digital world. However, I'm making a concerted effort to put devices down unless | |
| 79 | necessary and force myself to perform other activities in the physical world | |
| 80 | instead. | |
| 81 | ||
| 82 | I've started with a few basics that are always readily available to me: | |
| 83 | ||
| 84 | - Do a chore, such as organizing or cleaning. | |
| 85 | - Read a book, study a piece of art, etc. | |
| 86 | - Exercise or get outdoors. | |
| 87 | - Participate in a hobby, such as photography, birding, disc golf, etc. | |
| 88 | - Let yourself be bored and wander into creativity. | |
| 89 | ||
| 90 | ## Making Progress | |
| 91 | ||
| 92 | I'll be taking notes as I continue down this journey and hope to see positive | |
| 93 | trends. I've always been a minimalist in the physical world and it feels | |
| 94 | refreshing to filter out the clutter that has come to dominate my digital life | |
| 95 | over the years. | |
| 96 | ||
| 97 | I'm excited to see where this journey leads. | |
| 98 | ||
blog/2023-10-11-self-hosting-authelia.org added +446
| @@ -0,0 +1,446 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-10-11T02:53:29+00:00 | |
| 3 | title = "Self-Hosting Authelia" | |
| 4 | description = "A simple tutorial to get Authelia running on a self-hosted server." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | [Authelia](https://www.authelia.com/) is an open-source authentication service | |
| 10 | that allows you to place a portal between end users on the internet and | |
| 11 | self-hosted services on your server. | |
| 12 | ||
| 13 | You can require one factor (username+password) or two factor authentication for | |
| 14 | any such user before allowing them to access a specific service on your domain. | |
| 15 | ||
| 16 | This guide will walk through a standard installation of Authelia for | |
| 17 | `example.com`, using `auth.example.com` as Authelia's authentication domain and | |
| 18 | `teddit.example.com` as the website we want to protect behind the authentication | |
| 19 | portal. | |
| 20 | ||
| 21 | ## Prerequisites | |
| 22 | ||
| 23 | This guide assumes you have the following already set-up: | |
| 24 | ||
| 25 | - A registered domain with DNS pointing to your server. | |
| 26 | - A subdomain for Authelia (`auth.example.com`) and a subdomain to protect via | |
| 27 | Authelia (`app.example.com`). | |
| 28 | - A working Nginx web server. | |
| 29 | - Docker and docker-compose installed. | |
| 30 | ||
| 31 | ## Installation | |
| 32 | ||
| 33 | This guide will walk through each installation step one-by-one, starting with | |
| 34 | the container and finishing by cleaning up external access via an Nginx reverse | |
| 35 | proxy. | |
| 36 | ||
| 37 | ### Docker-Compose | |
| 38 | ||
| 39 | To start, create a directory for Authelia and create a `docker-compose.yml` | |
| 40 | file. | |
| 41 | ||
| 42 | ```sh | |
| 43 | mkdir ~/authelia | |
| 44 | nano ~/authelia/docker-compose.yml | |
| 45 | ``` | |
| 46 | ||
| 47 | Within this file, paste the following content. If you prefer a different local | |
| 48 | port, modify the port on the left side of the colon on the `9091:9091` line. Be | |
| 49 | sure to modify the `TZ` variable to your timezone. | |
| 50 | ||
| 51 | ```yml | |
| 52 | version: '3.3' | |
| 53 | ||
| 54 | services: | |
| 55 | authelia: | |
| 56 | image: authelia/authelia | |
| 57 | container_name: authelia | |
| 58 | volumes: | |
| 59 | - ./config:/config | |
| 60 | ports: | |
| 61 | - 9091:9091 | |
| 62 | environment: | |
| 63 | - TZ=America/Chicago | |
| 64 | ``` | |
| 65 | ||
| 66 | Start the container with docker-compose: | |
| 67 | ||
| 68 | ```sh | |
| 69 | sudo docker-compose up -d | |
| 70 | ``` | |
| 71 | ||
| 72 | After the first start, the container will automatically exit and require you to | |
| 73 | modify the app's configuration files before continuing. Read on to learn more. | |
| 74 | ||
| 75 | ### Authelia Configuration | |
| 76 | ||
| 77 | To configure Authelia before we restart the container, we need to open the | |
| 78 | `config` directory and modify the files. Start by editing the | |
| 79 | `configuration.yml` file, where all of Authelia's settings are stored. | |
| 80 | ||
| 81 | My personal preference is to copy the original configuration file to a backup | |
| 82 | file and edit a fresh copy. | |
| 83 | ||
| 84 | ```sh | |
| 85 | sudo cp ~/authelia/config/configuration.yml ~/authelia/config/configuration.yml.bk | |
| 86 | sudo nano ~/authelia/config/configuration.yml | |
| 87 | ``` | |
| 88 | ||
| 89 | Within the blank `configuration.yml` file, paste the following information. You | |
| 90 | will need to make quite a few updates, so be sure to read each line carefully | |
| 91 | and modify as necessary. | |
| 92 | ||
| 93 | The major required changes are: | |
| 94 | ||
| 95 | - Any instances of `example.com` should be replaced by your domain. | |
| 96 | - `jwt_secret` - Use the `pwgen 40 1` command to generate a secret for yourself. | |
| 97 | - `access_control` - Set the Authelia domain to bypass here, as well as any | |
| 98 | subdomains you want to protect. | |
| 99 | - `session` > `secret` - Use the `pwgen 40 1` command to generate a secret for | |
| 100 | yourself. | |
| 101 | - `regulation` - Set the variables here to restrict login attempts and bans. | |
| 102 | - `storage` > `encryption_key` - Use the `pwgen 40 1` command to generate a | |
| 103 | secret for yourself. | |
| 104 | - `smtp` - If you have access to an SMTP service, set up the information here to | |
| 105 | active outgoing emails. | |
| 106 | ||
| 107 | ```yml | |
| 108 | # yamllint disable rule:comments-indentation | |
| 109 | --- | |
| 110 | ############################################################################### | |
| 111 | # Authelia Configuration # | |
| 112 | ############################################################################### | |
| 113 | ||
| 114 | theme: dark | |
| 115 | jwt_secret: aiS5iedaiv6eeVaideeLeich5roo6ohvaf3Vee1a # pwgen 40 1 | |
| 116 | ||
| 117 | default_redirection_url: https://example.com | |
| 118 | ||
| 119 | server: | |
| 120 | host: 0.0.0.0 | |
| 121 | port: 9091 | |
| 122 | path: "" | |
| 123 | read_buffer_size: 4096 | |
| 124 | write_buffer_size: 4096 | |
| 125 | enable_pprof: false | |
| 126 | enable_expvars: false | |
| 127 | disable_healthcheck: false | |
| 128 | tls: | |
| 129 | key: "" | |
| 130 | certificate: "" | |
| 131 | ||
| 132 | log: | |
| 133 | level: debug | |
| 134 | ||
| 135 | totp: | |
| 136 | issuer: example.com | |
| 137 | period: 30 | |
| 138 | skew: 1 | |
| 139 | ||
| 140 | authentication_backend: | |
| 141 | disable_reset_password: false | |
| 142 | refresh_interval: 5m | |
| 143 | file: | |
| 144 | path: /config/users_database.yml | |
| 145 | password: | |
| 146 | algorithm: argon2id | |
| 147 | iterations: 1 | |
| 148 | key_length: 32 | |
| 149 | salt_length: 16 | |
| 150 | memory: 1024 | |
| 151 | parallelism: 8 | |
| 152 | ||
| 153 | access_control: | |
| 154 | default_policy: deny | |
| 155 | rules: | |
| 156 | - domain: | |
| 157 | - "auth.example.com" | |
| 158 | policy: bypass | |
| 159 | - domain: "teddit.example.com" | |
| 160 | policy: one_factor | |
| 161 | ||
| 162 | session: | |
| 163 | name: authelia_session | |
| 164 | secret: aiS5iedaiv6eeVaideeLeich5roo6ohvaf3Vee1a # pwgen 40 1 | |
| 165 | expiration: 3600 | |
| 166 | inactivity: 300 | |
| 167 | domain: example.com | |
| 168 | ||
| 169 | regulation: | |
| 170 | max_retries: 5 | |
| 171 | find_time: 10m | |
| 172 | ban_time: 12h | |
| 173 | ||
| 174 | storage: | |
| 175 | local: | |
| 176 | path: /config/db.sqlite3 | |
| 177 | encryption_key: aiS5iedaiv6eeVaideeLeich5roo6ohvaf3Vee1a # pwgen 40 1 | |
| 178 | ||
| 179 | notifier: | |
| 180 | disable_startup_check: true | |
| 181 | smtp: | |
| 182 | username: user@example.com | |
| 183 | password: password | |
| 184 | host: smtp.example.com | |
| 185 | port: 465 | |
| 186 | sender: user@example.com | |
| 187 | identifier: example.com | |
| 188 | subject: "[Authelia] {title}" | |
| 189 | startup_check_address: user@example.com | |
| 190 | disable_require_tls: false | |
| 191 | disable_html_emails: true | |
| 192 | tls: | |
| 193 | skip_verify: false | |
| 194 | minimum_version: TLS1.2 | |
| 195 | ... | |
| 196 | ``` | |
| 197 | ||
| 198 | ### Authelia Users | |
| 199 | ||
| 200 | Next, create the users file for authentication. | |
| 201 | ||
| 202 | ```sh | |
| 203 | sudo nano ~/authelia/config/users_database.yml | |
| 204 | ``` | |
| 205 | ||
| 206 | Within the file, you will need to create an entry for each user that needs | |
| 207 | access to Authelia. The `my_username` entry will be the username used on the | |
| 208 | login page. | |
| 209 | ||
| 210 | To generate the password, go to [Argon2 Hash Generator](https://argon2.online), | |
| 211 | generate a random salt, and make sure the rest of the settings match the | |
| 212 | `authentication_backend` section of `configuration.yml` file. | |
| 213 | ||
| 214 | ```yml | |
| 215 | users: | |
| 216 | my_username: | |
| 217 | displayname: "My User" | |
| 218 | # Generated at https://argon2.online/ -- match the settings in | |
| 219 | # the `authentication_backend` section of configuration.yml | |
| 220 | password: "" | |
| 221 | email: email@example.com | |
| 222 | groups: | |
| 223 | - admins | |
| 224 | - dev | |
| 225 | ``` | |
| 226 | ||
| 227 | Once the app is configured, restart the container from scratch. | |
| 228 | ||
| 229 | ```sh | |
| 230 | cd ~/authelia | |
| 231 | sudo docker-compose down && sudo docker-compose up -d | |
| 232 | ``` | |
| 233 | ||
| 234 | ### Nginx: Authelia Domain | |
| 235 | ||
| 236 | Once the container is running and configured, the final step is to configure | |
| 237 | external access to the server via Nginx reverse proxy. | |
| 238 | ||
| 239 | Start by creating the Authelia domain. | |
| 240 | ||
| 241 | ```sh | |
| 242 | sudo nano /etc/nginx/sites-available/auth | |
| 243 | ``` | |
| 244 | ||
| 245 | Within this file, paste the following information and be sure to update | |
| 246 | `example.com` to your domain. Make sure the `$upstream_authelia` variable | |
| 247 | matches the location of your Authelia container. | |
| 248 | ||
| 249 | ```conf | |
| 250 | server { | |
| 251 | if ($host ~ ^[^.]+\.example\.com$) { | |
| 252 | return 301 https://$host$request_uri; | |
| 253 | } | |
| 254 | ||
| 255 | listen [::]:80; | |
| 256 | listen 80; | |
| 257 | server_name auth.example.com; | |
| 258 | return 404; | |
| 259 | } | |
| 260 | ||
| 261 | server { | |
| 262 | listen [::]:443 ssl http2; | |
| 263 | listen 443 ssl http2; | |
| 264 | server_name auth.example.com; | |
| 265 | access_log /var/log/nginx/auth.access.log; | |
| 266 | error_log /var/log/nginx/auth.error.log; | |
| 267 | ||
| 268 | ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; | |
| 269 | ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; | |
| 270 | include /etc/letsencrypt/options-ssl-nginx.conf; | |
| 271 | ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; | |
| 272 | ||
| 273 | location / { | |
| 274 | set $upstream_authelia http://127.0.0.1:9091; | |
| 275 | proxy_pass $upstream_authelia; | |
| 276 | client_body_buffer_size 128k; | |
| 277 | ||
| 278 | proxy_next_upstream error timeout invalid_header http_500 http_502 http_503; | |
| 279 | ||
| 280 | send_timeout 5m; | |
| 281 | proxy_read_timeout 360; | |
| 282 | proxy_send_timeout 360; | |
| 283 | proxy_connect_timeout 360; | |
| 284 | ||
| 285 | proxy_set_header Host $host; | |
| 286 | proxy_set_header X-Real-IP $remote_addr; | |
| 287 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
| 288 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 289 | proxy_set_header X-Forwarded-Host $http_host; | |
| 290 | proxy_set_header X-Forwarded-Uri $request_uri; | |
| 291 | proxy_set_header X-Forwarded-Ssl on; | |
| 292 | proxy_redirect http:// $scheme://; | |
| 293 | proxy_http_version 1.1; | |
| 294 | proxy_set_header Connection ""; | |
| 295 | proxy_cache_bypass $cookie_session; | |
| 296 | proxy_no_cache $cookie_session; | |
| 297 | proxy_buffers 64 256k; | |
| 298 | } | |
| 299 | ||
| 300 | } | |
| 301 | ``` | |
| 302 | ||
| 303 | Next, symlink the file and restart Nginx. If there are errors, be sure to | |
| 304 | resolve those before moving on. | |
| 305 | ||
| 306 | ```sh | |
| 307 | sudo ln -s /etc/nginx/sites-available/auth /etc/nginx/sites-enabled/auth | |
| 308 | sudo systemctl restart nginx.service | |
| 309 | ``` | |
| 310 | ||
| 311 | ### Nginx: Protected Domain(s) | |
| 312 | ||
| 313 | Now that Authelia is accessible externally, you need to configure the domain you | |
| 314 | intend to protect with Authelia. In this example, I'm protecting | |
| 315 | `teddit.example.com`. | |
| 316 | ||
| 317 | Similar to the process above, paste the content and update the relevant | |
| 318 | variables. | |
| 319 | ||
| 320 | ```sh | |
| 321 | sudo nano /etc/nginx/sites-available/teddit | |
| 322 | ``` | |
| 323 | ||
| 324 | ```conf | |
| 325 | server { | |
| 326 | if ($host ~ ^[^.]+\.example\.com$) { | |
| 327 | return 301 https://$host$request_uri; | |
| 328 | } | |
| 329 | ||
| 330 | listen [::]:80; | |
| 331 | listen 80; | |
| 332 | server_name teddit.example.com; | |
| 333 | return 404; | |
| 334 | } | |
| 335 | ||
| 336 | server { | |
| 337 | listen [::]:443 ssl http2; | |
| 338 | listen 443 ssl http2; | |
| 339 | server_name teddit.example.com; | |
| 340 | access_log /var/log/nginx/teddit.access.log; | |
| 341 | error_log /var/log/nginx/teddit.error.log; | |
| 342 | ||
| 343 | add_header X-Content-Type-Options "nosniff"; | |
| 344 | add_header X-XSS-Protection "1; mode=block"; | |
| 345 | add_header X-Frame-Options "DENY"; | |
| 346 | add_header Strict-Transport-Security "max-age=63072000; includeSubDomains"; | |
| 347 | add_header Referrer-Policy "no-referrer"; | |
| 348 | ||
| 349 | ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; | |
| 350 | ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; | |
| 351 | include /etc/letsencrypt/options-ssl-nginx.conf; | |
| 352 | ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; | |
| 353 | ||
| 354 | location /authelia { | |
| 355 | internal; | |
| 356 | set $upstream_authelia http://127.0.0.1:9091/api/verify; | |
| 357 | proxy_pass_request_body off; | |
| 358 | proxy_pass $upstream_authelia; | |
| 359 | proxy_set_header Content-Length ""; | |
| 360 | ||
| 361 | proxy_next_upstream error timeout invalid_header http_500 http_502 http_503; | |
| 362 | client_body_buffer_size 128k; | |
| 363 | proxy_set_header Host $host; | |
| 364 | proxy_set_header X-Original-URL $scheme://$http_host$request_uri; | |
| 365 | proxy_set_header X-Real-IP $remote_addr; | |
| 366 | proxy_set_header X-Forwarded-For $remote_addr; | |
| 367 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 368 | proxy_set_header X-Forwarded-Host $http_host; | |
| 369 | proxy_set_header X-Forwarded-Uri $request_uri; | |
| 370 | proxy_set_header X-Forwarded-Ssl on; | |
| 371 | proxy_redirect http:// $scheme://; | |
| 372 | proxy_http_version 1.1; | |
| 373 | proxy_set_header Connection ""; | |
| 374 | proxy_cache_bypass $cookie_session; | |
| 375 | proxy_no_cache $cookie_session; | |
| 376 | proxy_buffers 4 32k; | |
| 377 | ||
| 378 | send_timeout 5m; | |
| 379 | proxy_read_timeout 240; | |
| 380 | proxy_send_timeout 240; | |
| 381 | proxy_connect_timeout 240; | |
| 382 | } | |
| 383 | ||
| 384 | location / { | |
| 385 | set $upstream_teddit http://127.0.0.1:8686; | |
| 386 | proxy_pass $upstream_teddit; | |
| 387 | ||
| 388 | auth_request /authelia; | |
| 389 | auth_request_set $target_url https://$http_host$request_uri; | |
| 390 | auth_request_set $user $upstream_http_remote_user; | |
| 391 | auth_request_set $email $upstream_http_remote_email; | |
| 392 | auth_request_set $groups $upstream_http_remote_groups; | |
| 393 | proxy_set_header Remote-User $user; | |
| 394 | proxy_set_header Remote-Email $email; | |
| 395 | proxy_set_header Remote-Groups $groups; | |
| 396 | ||
| 397 | error_page 401 =302 https://auth.example.com/?rd=$target_url; | |
| 398 | ||
| 399 | client_body_buffer_size 128k; | |
| 400 | ||
| 401 | proxy_next_upstream error timeout invalid_header http_500 http_502 http_503; | |
| 402 | ||
| 403 | send_timeout 5m; | |
| 404 | proxy_read_timeout 360; | |
| 405 | proxy_send_timeout 360; | |
| 406 | proxy_connect_timeout 360; | |
| 407 | ||
| 408 | proxy_set_header Host $host; | |
| 409 | proxy_set_header Upgrade $http_upgrade; | |
| 410 | proxy_set_header Connection upgrade; | |
| 411 | proxy_set_header Accept-Encoding gzip; | |
| 412 | proxy_set_header X-Real-IP $remote_addr; | |
| 413 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
| 414 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 415 | proxy_set_header X-Forwarded-Host $http_host; | |
| 416 | proxy_set_header X-Forwarded-Uri $request_uri; | |
| 417 | proxy_set_header X-Forwarded-Ssl on; | |
| 418 | proxy_redirect http:// $scheme://; | |
| 419 | proxy_http_version 1.1; | |
| 420 | proxy_set_header Connection ""; | |
| 421 | proxy_cache_bypass $cookie_session; | |
| 422 | proxy_no_cache $cookie_session; | |
| 423 | proxy_buffers 64 256k; | |
| 424 | } | |
| 425 | } | |
| 426 | ``` | |
| 427 | ||
| 428 | Same as before, symlink the file and restart Nginx. | |
| 429 | ||
| 430 | ```sh | |
| 431 | sudo ln -s /etc/nginx/sites-available/teddit /etc/nginx/sites-enabled/teddit | |
| 432 | sudo systemctl restart nginx.service | |
| 433 | ``` | |
| 434 | ||
| 435 | ## Results | |
| 436 | ||
| 437 | When visiting the protected domain, you will now be redirected to your | |
| 438 | authentication domain and presented with the Authelia login portal. | |
| 439 | ||
| 440 |  | |
| 441 | ||
| 442 | Once you've successfully authenticated, you can visit your authentication domain | |
| 443 | directly and see that you're currently authenticated to any domain protected by | |
| 444 | Authelia. | |
| 445 | ||
| 446 |  | |
blog/2023-10-15-alpine-ssh-hardening.org added +60
| @@ -0,0 +1,60 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-10-16T00:20:45+00:00 | |
| 3 | title = "SSH Hardening for Alpine Linux" | |
| 4 | description = "A quick guide to hardening SSH on Alpine Linux." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | This guide follows the standard | |
| 10 | [ssh-audit](https://www.ssh-audit.com/hardening_guides.html) hardening guide, | |
| 11 | tweaked for Alpine Linux. | |
| 12 | ||
| 13 | ## Hardening Guide | |
| 14 | ||
| 15 | These steps must be performed as root. You can try to use `doas` or `sudo`, | |
| 16 | but there may be issues. | |
| 17 | ||
| 18 | 1. Re-generate the RSA and ED25519 keys | |
| 19 | ```sh | |
| 20 | rm /etc/ssh/ssh_host_* | |
| 21 | ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N "" | |
| 22 | ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N "" | |
| 23 | ``` | |
| 24 | 2. Remove small Diffie-Hellman moduli | |
| 25 | ```sh | |
| 26 | awk '$5 >= 3071' /etc/ssh/moduli > /etc/ssh/moduli.safe | |
| 27 | mv /etc/ssh/moduli.safe /etc/ssh/moduli | |
| 28 | ``` | |
| 29 | 3. Enable the RSA and ED25519 HostKey directives in the /etc/ssh/sshd_config | |
| 30 | file | |
| 31 | ```sh | |
| 32 | sed -i 's/^\#HostKey \/etc\/ssh\/ssh_host_\(rsa\|ed25519\)_key$/HostKey \/etc\/ssh\/ssh_host_\1_key/g' /etc/ssh/sshd_config | |
| 33 | ``` | |
| 34 | 4. Restrict supported key exchange, cipher, and MAC algorithms | |
| 35 | ```sh | |
| 36 | echo -e "\n# Restrict key exchange, cipher, and MAC algorithms, as per sshaudit.com\n# hardening guide.\nKexAlgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha256\nCiphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr\nMACs hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com\nHostKeyAlgorithms ssh-ed25519,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256,rsa-sha2-256-cert-v01@openssh.com" > /etc/ssh/sshd_config.d/ssh-audit_hardening.conf | |
| 37 | ``` | |
| 38 | 5. Include the /etc/ssh/sshd_config.d directory | |
| 39 | ```sh | |
| 40 | echo -e "Include /etc/ssh/sshd_config.d/*.conf" > /etc/ssh/sshd_config | |
| 41 | ``` | |
| 42 | 6. Restart OpenSSH server | |
| 43 | ```sh | |
| 44 | rc-service sshd restart | |
| 45 | ``` | |
| 46 | ||
| 47 | ## Testing SSH | |
| 48 | ||
| 49 | You can test the results with the `ssh-audit` python script. | |
| 50 | ||
| 51 | ```sh | |
| 52 | pip3 install ssh-audit | |
| 53 | ssh-audit localhost | |
| 54 | ``` | |
| 55 | ||
| 56 | If everything succeeded, the results will show as all green. If anything is | |
| 57 | yellow, orange, or red, you may need to tweak additional settings. | |
| 58 | ||
| 59 |  | |
blog/2023-10-17-self-hosting-anonymousoverflow.org added +131
| @@ -0,0 +1,131 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-10-17T15:44:45.601627917+00:00 | |
| 3 | title = "Self-Hosting AnonymousOverflow" | |
| 4 | description = "A quick guide to self-hosting AnonymousOverflow." | |
| 5 | +++ | |
| 6 | ||
| 7 | ## Overview | |
| 8 | ||
| 9 | I recently launched an instance of AnonymousOverflow at | |
| 10 | [ao.0x4b1d.org](https://ao.0x4b1d.org) and wanted to write a brief post on how | |
| 11 | easy it is to install with Docker Compose and Nginx. | |
| 12 | ||
| 13 | This guide uses Ubuntu server, Docker Compose, and Nginx as a reverse proxy. | |
| 14 | ||
| 15 | ## Installation | |
| 16 | ||
| 17 | ### Docker Compose | |
| 18 | ||
| 19 | To install AnonymousOverflow, start by creating a directory for the application | |
| 20 | and create its `docker-compose.yml` file. | |
| 21 | ||
| 22 | ```sh | |
| 23 | mkdir ~/anonymousoverflow && cd ~/anonymousoverflow | |
| 24 | nano docker-compose.yml | |
| 25 | ``` | |
| 26 | ||
| 27 | Within this file, paste the following information. Be sure to change the | |
| 28 | `APP_URL`, `JWT_SIGNING_SECRET`, and `ports` to match your needs. | |
| 29 | ||
| 30 | ```yaml | |
| 31 | version: '3' | |
| 32 | ||
| 33 | services: | |
| 34 | anonymousoverflow: | |
| 35 | container_name: 'app' | |
| 36 | image: 'ghcr.io/httpjamesm/anonymousoverflow:release' | |
| 37 | environment: | |
| 38 | - APP_URL=https://ao.example.com | |
| 39 | - JWT_SIGNING_SECRET=secret #pwgen 40 1 | |
| 40 | ports: | |
| 41 | - '9380:8080' | |
| 42 | restart: 'always' | |
| 43 | ``` | |
| 44 | ||
| 45 | Save and exit the file when complete. You can now launch the container and | |
| 46 | access it via your local network. | |
| 47 | ||
| 48 | ```sh | |
| 49 | sudo docker-compose up -d | |
| 50 | ``` | |
| 51 | ||
| 52 | ### Nginx Reverse Proxy | |
| 53 | ||
| 54 | If you want to access this service outside the local network, I recommend using | |
| 55 | Nginx as a reverse proxy. | |
| 56 | ||
| 57 | Let's start by creating a configuration file. | |
| 58 | ||
| 59 | ```sh | |
| 60 | sudo nano /etc/nginx/sites-available/ao | |
| 61 | ``` | |
| 62 | ||
| 63 | Within this file, paste the following content and repace `ao.example.com` with | |
| 64 | your URL. You may need to update the SSL certificate statements if your | |
| 65 | certificates are in a different location. | |
| 66 | ||
| 67 | ```conf | |
| 68 | server { | |
| 69 | if ($host ~ ^[^.]+\.cleberg\.net$) { | |
| 70 | return 301 https://$host$request_uri; | |
| 71 | } | |
| 72 | ||
| 73 | listen [::]:80; | |
| 74 | listen 80; | |
| 75 | server_name ao.example.com; | |
| 76 | return 404; | |
| 77 | } | |
| 78 | ||
| 79 | server { | |
| 80 | listen [::]:443 ssl http2; | |
| 81 | listen 443 ssl http2; | |
| 82 | server_name ao.example.com; | |
| 83 | access_log /var/log/nginx/ao.access.log; | |
| 84 | error_log /var/log/nginx/ao.error.log; | |
| 85 | ||
| 86 | add_header X-Content-Type-Options "nosniff"; | |
| 87 | add_header X-XSS-Protection "1; mode=block"; | |
| 88 | add_header X-Frame-Options "DENY"; | |
| 89 | add_header Strict-Transport-Security "max-age=63072000; includeSubDomains"; | |
| 90 | add_header Referrer-Policy "no-referrer"; | |
| 91 | ||
| 92 | ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; | |
| 93 | ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; | |
| 94 | include /etc/letsencrypt/options-ssl-nginx.conf; | |
| 95 | ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; | |
| 96 | ||
| 97 | location / { | |
| 98 | set $upstream_ao http://127.0.0.1:9380; | |
| 99 | proxy_pass $upstream_ao; | |
| 100 | ||
| 101 | proxy_set_header Host $host; | |
| 102 | proxy_set_header Upgrade $http_upgrade; | |
| 103 | proxy_set_header Connection upgrade; | |
| 104 | proxy_set_header Accept-Encoding gzip; | |
| 105 | proxy_set_header X-Real-IP $remote_addr; | |
| 106 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | |
| 107 | proxy_set_header X-Forwarded-Proto $scheme; | |
| 108 | proxy_set_header X-Forwarded-Host $http_host; | |
| 109 | proxy_set_header X-Forwarded-Uri $request_uri; | |
| 110 | proxy_set_header X-Forwarded-Ssl on; | |
| 111 | proxy_redirect http:// $scheme://; | |
| 112 | proxy_http_version 1.1; | |
| 113 | proxy_set_header Connection ""; | |
| 114 | proxy_cache_bypass $cookie_session; | |
| 115 | proxy_no_cache $cookie_session; | |
| 116 | proxy_buffers 64 256k; | |
| 117 | } | |
| 118 | } | |
| 119 | ``` | |
| 120 | ||
| 121 | Save and exit the file when complete. On Ubuntu, you will need to symlink the | |
| 122 | configuration file before it will be recognized by Nginx. Once complete, | |
| 123 | simply restart the web server. | |
| 124 | ||
| 125 | ```sh | |
| 126 | sudo ln -s /etc/nginx/sites-available/ao /etc/nginx/sites-enabled/ao | |
| 127 | sudo systemctl restart nginx.service | |
| 128 | ``` | |
| 129 | ||
| 130 | The website will now be available publicly. Visit | |
| 131 | [my instance](https://ao.0x4b1d.org) for an example. | |
blog/2023-11-08-scli.org added +146
| @@ -0,0 +1,146 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-11-08T21:37:00.307616604+00:00 | |
| 3 | title = "Installing scli on Alpine Linux (musl)" | |
| 4 | description = "" | |
| 5 | +++ | |
| 6 | ||
| 7 | [scli](https://github.com/isamert/scli) is a command-line tool that allows you | |
| 8 | to connect to your Signal messenger account. This program utilizes a two-pane | |
| 9 | display that shows you chats on the left and the focused conversation on the | |
| 10 | right. | |
| 11 | ||
| 12 | This guide will show you how to install `scli` and its dependencies on Alpine | |
| 13 | Linux, which requires some extra work due to musl. | |
| 14 | ||
| 15 | If you're using a non-musl system, you can likely following the `scli` README | |
| 16 | and download the packaged binaries for an easier installation process. | |
| 17 | ||
| 18 | ## Dependencies | |
| 19 | ||
| 20 | In order to use `scli`, you need a few dependencies: | |
| 21 | ||
| 22 | - `openjdk17-jre` - Used as a dependency for the `signal-cli` tool. Version may | |
| 23 | vary. | |
| 24 | - `signal-cli` - Used as the backbone of the `scli` tool. | |
| 25 | - `findutils` - Replaces the standard Busybox version of `xargs`. | |
| 26 | - `urwid` - A console user interface library for Python. | |
| 27 | - `urwid-readline` - For GNU emacs-like keybinds on the input line. | |
| 28 | - `qrencode` - Displays a QR code in the terminal to link the device using your | |
| 29 | phone. Not necessary if you're only linking on desktop and can copy/paste the | |
| 30 | connection URL. | |
| 31 | ||
| 32 | Let's start by installing the packages available via Alpine's repositories. Be | |
| 33 | sure to install the latest version of `openjdk`. If you run into Java-related | |
| 34 | issues, uninstall `openjdk` and install an older version. | |
| 35 | ||
| 36 | ```sh | |
| 37 | doas apk add openjdk17-jre findutils qrencode | |
| 38 | ``` | |
| 39 | ||
| 40 | Next, let's install `signal-cli`. Be sure to export the version of `signal-cli` | |
| 41 | that you want. I use version `0.12.4` below, but that may be outdated by the | |
| 42 | time you're reading this. | |
| 43 | ||
| 44 | ```sh | |
| 45 | export VERSION="0.12.4" | |
| 46 | wget https://github.com/AsamK/signal-cli/releases/download/v"${VERSION}"/signal-cli-"${VERSION}".tar.gz | |
| 47 | doas tar xf signal-cli-"${VERSION}".tar.gz -C /opt | |
| 48 | doas ln -sf /opt/signal-cli-${VERSION}"/bin/signal/cli /usr/local/bin | |
| 49 | ``` | |
| 50 | ||
| 51 | Finally, install the `urwid` packages using the Python packaging utility. | |
| 52 | ||
| 53 | ```sh | |
| 54 | pip3 install urwid urwid-readline | |
| 55 | ``` | |
| 56 | ||
| 57 | ## Installation | |
| 58 | ||
| 59 | Now that we have all of the dependencies we need, we can install `scli`. Start | |
| 60 | by simply cloning the repository. | |
| 61 | ||
| 62 | ```sh | |
| 63 | git clone https://github.com/isamert/scli | |
| 64 | ``` | |
| 65 | ||
| 66 | When I cloned this repository on 2023-11-08, I found a bug in the logic that | |
| 67 | required a fix. You must edit the `scli` file and replace the one instance of | |
| 68 | `RLIMIT_OFILE` with `RLIMIT_NOFILE`. | |
| 69 | ||
| 70 | ```sh | |
| 71 | cd scli | |
| 72 | nano scli | |
| 73 | ``` | |
| 74 | ||
| 75 | Once complete, you can move this program to anywhere on your `$PATH`. I chose | |
| 76 | the following directory. | |
| 77 | ||
| 78 | ```sh | |
| 79 | doas mv scli /usr/local/bin/scli | |
| 80 | ``` | |
| 81 | ||
| 82 | ## Initial Setup | |
| 83 | ||
| 84 | Now that everything is installed, we can login and configure the client. Start | |
| 85 | by generating a connection link. | |
| 86 | ||
| 87 | ```sh | |
| 88 | signal-cli link -n "YOUR-DEVICE-NICKNAME" | tee >(xargs -L 1 qrencode -t utf8) | |
| 89 | ``` | |
| 90 | ||
| 91 | This will generate a connection link and related QR code for you to use to link | |
| 92 | the devices together. Once complete, **wait patiently** for the connection | |
| 93 | process to finish. | |
| 94 | ||
| 95 | Once it completes, it will exit and return you to the prompt. From here, you | |
| 96 | need to perform an initial `receive` command to start things off. The `USERNAME` | |
| 97 | variable should be your phone number, such as `+15551237890`. | |
| 98 | ||
| 99 | ```sh | |
| 100 | signal-cli -u USERNAME receive | |
| 101 | ``` | |
| 102 | ||
| 103 | Also be sure to test the daemon to ensure it works properly. If no errors occur, | |
| 104 | it's working. If you run into errors because you're not running a DBUS session, | |
| 105 | see my notes below. | |
| 106 | ||
| 107 | ```sh | |
| 108 | signal-cli -u USERNAME daemon | |
| 109 | ``` | |
| 110 | ||
| 111 | Once the initial reception is complete, you are ready to use `scli`. | |
| 112 | ||
| 113 | This process will differ depending on your desktop environment (DE). If you are | |
| 114 | running a DE, you likely have a DBUS session running already and can simply | |
| 115 | launch the program. | |
| 116 | ||
| 117 | However, if you're like me and running your computer straight on the TTY without | |
| 118 | a DE, you'll need to start a DBUS session for this program. | |
| 119 | ||
| 120 | ```sh | |
| 121 | # If you're not running a DBUS session yet, you need to start one for scli | |
| 122 | dbus-run-session -- scli | |
| 123 | ||
| 124 | # OR - If you're already running a DBUS session, simply run scli | |
| 125 | scli | |
| 126 | ``` | |
| 127 | ||
| 128 | ## Configuration | |
| 129 | ||
| 130 | Lastly, there are a number of configuration options that you can pass via the | |
| 131 | command or in the `~/.config/sclirc` file. See the Github README for more | |
| 132 | information on configuration options. | |
| 133 | ||
| 134 | ```sh | |
| 135 | nano ~/.config/sclirc | |
| 136 | ``` | |
| 137 | ||
| 138 | ```conf | |
| 139 | # ~/.config/sclirc | |
| 140 | ||
| 141 | wrap-at = 80 | |
| 142 | enable-notifications = true | |
| 143 | ``` | |
| 144 | ||
| 145 | That's it! Following this guide, I have a functional `scli` program that | |
| 146 | successfully sends messages to my contacts and myself! | |
blog/2023-11-12-consolefonts.org added +5
| @@ -0,0 +1,5 @@ | ||
| 1 | +++ | |
| 2 | date = 2023-11-12T02:03:25.924298748+00:00 | |
| 3 | title = "How to Configure Console Fonts on Alpine Linux" | |
| 4 | description = "" | |
| 5 | +++ | |
blog/index.org added +127
| @@ -0,0 +1,127 @@ | ||
| 1 | #+title: Blog | |
| 2 | #+OPTIONS: toc:nil | |
| 3 | ||
| 4 | Use =C-f= to search blog post titles for keywords. | |
| 5 | ||
| 6 | * 2023 | |
| 7 | ||
| 8 | - 2023-11-08 [[./2023-11-08-scli.org][Installing scli on Alpine Linux (musl)]] | |
| 9 | - 2023-10-17 [[./2023-10-17-self-hosting-anonymousoverflow.org][Self-Hosting AnonymousOverflow]] | |
| 10 | - 2023-10-15 [[./2023-10-15-alpine-ssh-hardening.org][SSH Hardening for Alpine Linux]] | |
| 11 | - 2023-10-11 [[./2023-10-11-self-hosting-authelia.org][Self-Hosting Authelia]] | |
| 12 | - 2023-10-04 [[./2023-10-04-digital-minimalism.org][Digital Minimalism]] | |
| 13 | - 2023-09-19 [[./2023-09-19-audit-sql-scripts.org][Useful SQL Scripts for Auditing Logical Access]] | |
| 14 | - 2023-09-15 [[./2023-09-15-gitweb.org][Gitweb via Nginx]] | |
| 15 | - 2023-08-18 [[./2023-08-18-agile-auditing.org][Agile Auditing: An Introduction]] | |
| 16 | - 2023-07-19 [[./2023-07-19-plex-transcoder-errors.org][How to Fix Plex Error: 'Conversion failed. The transcoder failed to start up.']] | |
| 17 | - 2023-07-12 [[./2023-07-12-mullvad-wireguard-lan.org][Enabling LAN Access in Mullvad Wireguard Conf Files]] | |
| 18 | - 2023-06-30 [[./2023-06-30-self-hosting-voyager-lemmy-client.org][Self Hosting Voyager - A Lemmy Web Client]] | |
| 19 | - 2023-06-28 [[./2023-06-28-backblaze-b2.org][Getting Started with Backblaze B2 Cloud Storage]] | |
| 20 | - 2023-06-23 [[./2023-06-23-self-hosting-convos-irc.org][Self-Hosting Convos IRC Web Client]] | |
| 21 | - 2023-06-23 [[./2023-06-23-byobu.org][Byobu]] | |
| 22 | - 2023-06-20 [[./2023-06-20-audit-review-cheatsheet.org][Cheatsheet: Review Audit Test Results]] | |
| 23 | - 2023-06-18 [[./2023-06-18-unifi-ip-blocklist.org][Block IP Addresses and Subnets with Unifi Network Firewall]] | |
| 24 | - 2023-06-08 [[./2023-06-08-self-hosting-baikal-server.org][Self-Hosting Baïkal Server (CalDAV & CardDAV)]] | |
| 25 | - 2023-06-08 [[./2023-06-08-goaccess-geoip.org][Inspecting Nginx Logs with GoAccess and MaxMind GeoIP Data]] | |
| 26 | - 2023-05-22 [[./2023-05-22-burnout.org][Burnout]] | |
| 27 | - 2023-02-02 [[./2023-02-02-exploring-hare.org][Exploring the Hare Programming Language]] | |
| 28 | - 2023-01-28 [[./2023-01-28-self-hosting-wger-workout-manager.org][Self-Hosting Wger Workout Manager]] | |
| 29 | - 2023-01-23 [[./2023-01-23-random-mullvad-wireguard.org][Connecting to a Random Mullvad Wireguard Host]] | |
| 30 | - 2023-01-21 [[./2023-01-21-running-flatpak-apps-easily.org][Running Flatpak Apps with Symlinks]] | |
| 31 | - 2023-01-08 [[./2023-01-08-fedora-remove-login-manager.org][Remove the Login Manager from Fedora i3]] | |
| 32 | - 2023-01-05 [[./2023-01-05-mass-unlike-tumblr-posts.org][How to Easily Mass Unlike Tumblr Posts With Javascript]] | |
| 33 | - 2023-01-03 [[./2023-01-03-recent-website-changes.org][Recent Website Changes]] | |
| 34 | ||
| 35 | * 2022 | |
| 36 | ||
| 37 | - 2022-12-23 [[./2022-12-23-alpine-desktop.org][Alpine Linux as a Desktop OS]] | |
| 38 | - 2022-12-17 [[./2022-12-17-st.org][Simple Terminal]] | |
| 39 | - 2022-12-07 [[./2022-12-07-nginx-wildcard-redirect.org][Redirect Nginx Subdomains & Trailing Content with Regex]] | |
| 40 | - 2022-12-01 [[./2022-12-01-nginx-compression.org][Enable GZIP Compression on Nginx]] | |
| 41 | - 2022-11-29 [[./2022-11-29-nginx-referrer-ban-list.org][Creating a Referrer Ban List on Nginx]] | |
| 42 | - 2022-11-27 [[./2022-11-27-server-build.org][Building a Custom Rack-Mount Server]] | |
| 43 | - 2022-11-11 [[./2022-11-11-nginx-tmp-errors.org][Fixing Permission Errors in /var/lib/nginx]] | |
| 44 | - 2022-11-07 [[./2022-11-07-matrix-synapse.org][Self-Hosting Matrix Synapse on Alpine Linux]] | |
| 45 | - 2022-10-30 [[./2022-10-30-linux-display-manager.org][How to Disable or Change the Display Manager on Void Linux]] | |
| 46 | - 2022-10-22 [[./2022-10-22-alpine-linux.org][Alpine Linux: My New Server OS]] | |
| 47 | - 2022-10-20 [[./2022-10-20-syncthing.org][Syncthing: A Minimal Self-Hosted Cloud Storage Solution]] | |
| 48 | - 2022-10-04 [[./2022-10-04-mtp-linux.org][How to Mount an MTP Mobile Device on Linux]] | |
| 49 | - 2022-09-21 [[./2022-09-21-graphene-os.org][Installing Graphene OS on the Pixel 6 Pro]] | |
| 50 | - 2022-09-17 [[./2022-09-17-serenity-os.org][Serenity OS: Testing Out a Unique System]] | |
| 51 | - 2022-08-31 [[./2022-08-31-privacy.com-changes.org][Concerning Changes on Privacy.com]] | |
| 52 | - 2022-07-31 [[./2022-07-31-bash-it.org][Upgrade Bash with Bash-It & Ble.sh]] | |
| 53 | - 2022-07-30 [[./2022-07-30-flac-to-opus.org][Recursive Command-Line FLAC to Opus Conversion]] | |
| 54 | - 2022-07-25 [[./2022-07-25-curseradio.org][CurseRadio: Listening to the Radio on the Command Line]] | |
| 55 | - 2022-07-14 [[./2022-07-14-gnupg.org][GNU Privacy Guard (GPG)]] | |
| 56 | - 2022-07-01 [[./2022-07-01-git-server.org][Self-Hosting a Personal Git Server]] | |
| 57 | - 2022-06-24 [[./2022-06-24-fedora-i3.org][Rebooting My Love Affair with Linux]] | |
| 58 | - 2022-06-22 [[./2022-06-22-daily-poetry.org][Daily Plaintext Poetry via Email]] | |
| 59 | - 2022-06-16 [[./2022-06-16-terminal-lifestyle.org][A Terminal Lifestyle]] | |
| 60 | - 2022-06-07 [[./2022-06-07-freshrss.org][Self-Hosting FreshRSS]] | |
| 61 | - 2022-06-04 [[./2022-06-04-njalla-dns-api.org][Dynamic DNS with Njalla API]] | |
| 62 | - 2022-06-01 [[./2022-06-01-ditching-cloudflare.org][Ditching Cloudflare for Njalla]] | |
| 63 | - 2022-04-09 [[./2022-04-09-pinetime.org][PineTime: An Open-Source SmartWatch]] | |
| 64 | - 2022-04-02 [[./2022-04-02-nginx-reverse-proxy.org][Set-Up a Reverse Proxy with Nginx]] | |
| 65 | - 2022-03-26 [[./2022-03-26-ssh-mfa.org][Enable TOTP MFA for SSH]] | |
| 66 | - 2022-03-24 [[./2022-03-24-server-hardening.org][Hardening a Public-Facing Home Server]] | |
| 67 | - 2022-03-23 [[./2022-03-23-nextcloud-on-ubuntu.org][Installing Nextcloud on Ubuntu]] | |
| 68 | - 2022-03-23 [[./2022-03-23-cloudflare-dns-api.org][Dynamic DNS with Cloudflare API]] | |
| 69 | - 2022-03-08 [[./2022-03-08-plex-migration.org][Migrating Plex to New Server (+ Nvidia Transcoding)]] | |
| 70 | - 2022-03-03 [[./2022-03-03-financial-database.org][Maintaining a Personal Financial Database]] | |
| 71 | - 2022-03-02 [[./2022-03-02-note-taking.org][Easy, Reliable Note-Taking]] | |
| 72 | - 2022-02-22 [[./2022-02-22-tuesday.org][Tuesday]] | |
| 73 | - 2022-02-20 [[./2022-02-20-nginx-caching.org][Caching Static Content with Nginx]] | |
| 74 | - 2022-02-17 [[./2022-02-17-exiftool.org][Stripping Image Metadata with exiftool]] | |
| 75 | - 2022-02-16 [[./2022-02-16-debian-and-nginx.org][Migrating to a New Web Server Setup with Debian, Nginx, and Agate]] | |
| 76 | - 2022-02-10 [[./2022-02-10-leaving-the-office.org][Leaving Office-Based Work In the Past]] | |
| 77 | ||
| 78 | * 2021 | |
| 79 | ||
| 80 | - 2021-12-04 [[./2021-12-04-cisa.org][I Passed the CISA!]] | |
| 81 | - 2021-10-09 [[./2021-10-09-apache-redirect.org][Apache Redirect HTML Files to a Directory]] | |
| 82 | - 2021-08-25 [[./2021-08-25-audit-sampling.org][Audit Sampling with Python]] | |
| 83 | - 2021-07-15 [[./2021-07-15-delete-gitlab-repos.org][How to Delete All GitLab Repositories]] | |
| 84 | - 2021-05-30 [[./2021-05-30-changing-git-authors.org][Changing Git Authors]] | |
| 85 | - 2021-04-28 [[./2021-04-28-photography.org][Jumping Back Into Photography]] | |
| 86 | - 2021-04-23 [[./2021-04-23-php-comment-system.org][Roll Your Own Static Commenting System in PHP]] | |
| 87 | - 2021-04-17 [[./2021-04-17-gemini-server.org][Hosting a Gemini Server]] | |
| 88 | - 2021-03-30 [[./2021-03-30-vps-web-server.org][How to Set Up a VPS Web Server]] | |
| 89 | - 2021-03-28 [[./2021-03-28-vaporwave-vs-outrun.org][Vaporwave vs Outrun]] | |
| 90 | - 2021-03-28 [[./2021-03-28-gemini-capsule.org][Launching a Gemini Capsule]] | |
| 91 | - 2021-03-19 [[./2021-03-19-clone-github-repos.org][How to Clone All Repositories from a GitHub or Sourcehut Account]] | |
| 92 | - 2021-02-19 [[./2021-02-19-macos.org][macOS: Testing Out A New OS]] | |
| 93 | - 2021-01-07 [[./2021-01-07-ufw.org][Secure Your Network with the Uncomplicated Firewall]] | |
| 94 | - 2021-01-04 [[./2021-01-04-fediverse.org][A Simple Guide to the Fediverse]] | |
| 95 | - 2021-01-01 [[./2021-01-01-seum.org][SEUM: Speedrunners from Hell]] | |
| 96 | ||
| 97 | * 2020 | |
| 98 | ||
| 99 | - 2020-12-29 [[./2020-12-29-zork.org][Zork: Let's Explore a Classic]] | |
| 100 | - 2020-12-28 [[./2020-12-28-neon-drive.org][Neon Drive: A Nostalgic 80s Arcade Racing Game]] | |
| 101 | - 2020-12-27 [[./2020-12-27-website-redesign.org][Redesigning My Website: The 5 KB Result]] | |
| 102 | - 2020-10-12 [[./2020-10-12-mediocrity.org][On the Pursuit of Mediocrity]] | |
| 103 | - 2020-09-25 [[./2020-09-25-happiness-map.org][Data Visualization: World Choropleth Map of Happiness]] | |
| 104 | - 2020-09-22 [[./2020-09-22-internal-audit.org][What is Internal Audit?]] | |
| 105 | - 2020-09-01 [[./2020-09-01-visual-recognition.org][IBM Watson Visual Recognition]] | |
| 106 | - 2020-08-29 [[./2020-08-29-php-auth-flow.org][PHP Authentication Flow]] | |
| 107 | - 2020-08-22 [[./2020-08-22-redirect-github-pages.org][Redirect GitHub Pages from Subdomain to the Top-Level Domain]] | |
| 108 | - 2020-07-26 [[./2020-07-26-business-analysis.org][Algorithmically Analyzing Local Businesses]] | |
| 109 | - 2020-07-20 [[./2020-07-20-video-game-sales.org][Data Exploration: Video Game Sales]] | |
| 110 | - 2020-05-19 [[./2020-05-19-customizing-ubuntu.org][Beginner's Guide: Customizing Ubuntu]] | |
| 111 | - 2020-05-03 [[./2020-05-03-homelab.org][An Inside Look at My Homelab]] | |
| 112 | - 2020-03-25 [[./2020-03-25-session-messenger.org][Session Private Messenger]] | |
| 113 | - 2020-02-09 [[./2020-02-09-cryptography-basics.org][Cryptography Basics]] | |
| 114 | - 2020-01-26 [[./2020-01-26-steam-on-ntfs.org][Linux Gaming Tweak: Steam on NTFS Drives]] | |
| 115 | - 2020-01-25 [[./2020-01-25-linux-software.org][Linux Software]] | |
| 116 | ||
| 117 | * 2019 | |
| 118 | ||
| 119 | - 2019-12-16 [[./2019-12-16-password-security.org][Password Security]] | |
| 120 | - 2019-12-03 [[./2019-12-03-the-ansoff-matrix.org][The Ansoff Matrix]] | |
| 121 | - 2019-09-09 [[./2019-09-09-audit-analytics.org][Data Analysis in Auditing]] | |
| 122 | - 2019-01-07 [[./2019-01-07-useful-css.org][Useful CSS Snippets]] | |
| 123 | ||
| 124 | * 2018 | |
| 125 | ||
| 126 | - 2018-12-08 [[./2018-12-08-aes-encryption.org][AES Encryption]] | |
| 127 | - 2018-11-28 [[./2018-11-28-cpp-compiler.org][The C++ Compiler]] | |
index.org added +19
| @@ -0,0 +1,19 @@ | ||
| 1 | #+title: cleberg.net | |
| 2 | #+OPTIONS: toc:nil | |
| 3 | ||
| 4 | #+BEGIN_QUOTE | |
| 5 | Do AIs dream in electric sheep? | |
| 6 | ||
| 7 | He hoped it had nanosecond nightmares. | |
| 8 | #+END_QUOTE | |
| 9 | ||
| 10 | ** Pages | |
| 11 | ||
| 12 | - [[./blog/][Blog]] | |
| 13 | - [[./services/][Services]] | |
| 14 | - [[./wiki/][Wiki]] | |
| 15 | ||
| 16 | ** Contact | |
| 17 | ||
| 18 | - hello [at] cleberg.net | |
| 19 | - [[https://git.cleberg.net][Source Code]] | |
salary/index.org added +53
| @@ -0,0 +1,53 @@ | ||
| 1 | #+title: Salary | |
| 2 | #+date: 2023-10-01 | |
| 3 | ||
| 4 | * Salary Transparency | |
| 5 | ||
| 6 | The data below details the base salary information for each job I've held. This | |
| 7 | information is posted publicly to ensure others in my position have a solid | |
| 8 | reference point when determining if their current or proposed salary is | |
| 9 | appropriate. | |
| 10 | ||
| 11 | While sites like Glassdoor are locking salary data behind a paywall, LinkedIn is | |
| 12 | discontinuing LinkedIn Salary, and helpful websites like Big 4 Transparency are | |
| 13 | extremely rare, I wanted to provide my personal data publicly and freely to | |
| 14 | those who need it. | |
| 15 | ||
| 16 | I have seen what can happen when great employees don't know the market values | |
| 17 | for their skills and I happily help those in my teams, so I'm happy to extend | |
| 18 | this information to the online community. | |
| 19 | ||
| 20 | As a final note, there are numerous reasons that people in the same role are | |
| 21 | paid differently (expertise, years of experience, certifications, education, | |
| 22 | etc.) and that the data in this table should only be used as a single point of | |
| 23 | reference, not the whole story. | |
| 24 | ||
| 25 | * Salary Data | |
| 26 | ||
| 27 | Note: When in a role that gives periodic raises, I will create a new record with | |
| 28 | the new base salary in the table below. See the KPMG records for an example of a | |
| 29 | raise while in the same role. | |
| 30 | ||
| 31 | | Title | Company | Location | Start Date | End Date | Base Salary | | |
| 32 | | :-------------------------------------: | :----------------------: | :--------------: | :----------: | :--------: | :-----------: | | |
| 33 | | Senior Technology Assurance Associate | KPMG | Omaha, NE | 2023-10 | Current | $116,700 | | |
| 34 | | Senior Technology Assurance Associate | KPMG | Omaha, NE | 2022-06 | 2023-10 | $110,000 | | |
| 35 | | Senior Technology Risk Consultant | Ernst & Young | Des Moines, IA | 2021-09 | 2022-06 | $89,500 | | |
| 36 | | Senior IT Auditor | Ameritas | Lincoln, NE | 2021-05 | 2021-09 | $72,000 | | |
| 37 | | IT Auditor | Ameritas | Lincoln, NE | 2020-04 | 2021-05 | $65,000 | | |
| 38 | | IS Auditor II | Nelnet | Lincoln, NE | 2019-12 | 2020-04 | $58,000 | | |
| 39 | | IS Auditor I | Nelnet | Lincoln, NE | 2019-06 | 2019-12 | $20/hour | | |
| 40 | | Internal Audit Intern | Ameritas | Lincoln, NE | 2018-02 | 2019-06 | $16/hour | | |
| 41 | | Teaching Assistant | University of Nebraska | Lincoln, NE | 2017-08 | 2018-05 | $7/hour | | |
| 42 | | Community Management Intern | Walgreens | Lincoln, NE | 2017-06 | 2018-02 | $14/hour | | |
| 43 | ||
| 44 | This page was inspired by [[https://xeiaso.net/][Xe]], and I'm quoting the following wording from them as | |
| 45 | I want to reiterate this piece: | |
| 46 | ||
| 47 | #+BEGIN_QUOTE | |
| 48 | Please consider publishing your salary data like this as well. By open, | |
| 49 | voluntary transparency we can help to end stigmas around discussing pay and help | |
| 50 | ensure that the next generations of people in tech are treated fairly. Stigmas | |
| 51 | thrive in darkness but die in the light of day. You can help end the stigma by | |
| 52 | playing your cards out in the open like this. | |
| 53 | #+END_QUOTE | |
services/index.org added +16
| @@ -0,0 +1,16 @@ | ||
| 1 | #+title: Services | |
| 2 | #+OPTIONS: toc:nil | |
| 3 | ||
| 4 | - [[https://ao.cleberg.net][AnonymousOverflow]] - A StackOverflow proxy | |
| 5 | - [[https://cyberchef.cleberg.net][CyberChef]] - The Cyber Swiss Army Knife | |
| 6 | - [[https://files.cleberg.net][FileArchive]] - An interesting file archive | |
| 7 | - [[https://paste.cleberg.net][FlashPaper]] - One-time encrypted password/secret sharing | |
| 8 | - [[https://gh.cleberg.net][GotHub]] - An alternative front-end for GitHub | |
| 9 | - [[https://ip.cleberg.net][ifconfig.php]] - A "whatsmyip" tool | |
| 10 | - [[https://invidious.cleberg.net][Invidious]] - A YouTube proxy | |
| 11 | - [[https://nitter.cleberg.net][Nitter]] - A Twitter proxy | |
| 12 | - [[https://office.cleberg.net][Office]] - The world's smallest office suite | |
| 13 | - [[https://search.cleberg.net][SearXNG]] - A privacy-respecting, open metasearch engine | |
| 14 | ||
| 15 | See the [[https://git.cleberg.net/?p=cleberg.net.git][git log]] for this page if you want to see changes that have been made. | |
| 16 | ||
static/styles.css added +169
| @@ -0,0 +1,169 @@ | ||
| 1 | @media (prefers-reduced-motion:no-preference) { | |
| 2 | :root { | |
| 3 | scroll-behavior: smooth; | |
| 4 | } | |
| 5 | } | |
| 6 | ||
| 7 | body { | |
| 8 | background-color: hsl(0, 0%, 100%); | |
| 9 | color: hsl(0, 0%, 5%); | |
| 10 | font-family: -apple-system, BlinkMacSystemFont, avenir next, avenir, segoe ui, helvetica neue, helvetica, Cantarell, Ubuntu, roboto, noto, arial, sans-serif; | |
| 11 | font-size: 1rem; | |
| 12 | line-height: 1.5; | |
| 13 | max-width: 45em; | |
| 14 | margin: 0 auto; | |
| 15 | padding: 0 1rem; | |
| 16 | } | |
| 17 | ||
| 18 | a, a:visited { | |
| 19 | color: hsl(240, 100%, 35%); | |
| 20 | text-decoration: none; | |
| 21 | } | |
| 22 | ||
| 23 | a:hover { | |
| 24 | background-color: hsl(240, 100%, 35%); | |
| 25 | color: hsl(0, 0%, 100%); | |
| 26 | } | |
| 27 | ||
| 28 | #preamble { | |
| 29 | padding: 1rem 0; | |
| 30 | } | |
| 31 | ||
| 32 | #preamble nav a { | |
| 33 | margin-right: 0.5rem; | |
| 34 | } | |
| 35 | ||
| 36 | #postamble { | |
| 37 | border-top: 1px solid #000; | |
| 38 | margin-top: 1rem; | |
| 39 | } | |
| 40 | ||
| 41 | img { | |
| 42 | width: 100%; | |
| 43 | } | |
| 44 | ||
| 45 | table { | |
| 46 | border-collapse: collapse; | |
| 47 | } | |
| 48 | ||
| 49 | thead, th, tr, td { | |
| 50 | border: 1px solid hsl(0, 100%, 5%); | |
| 51 | padding: 0.25rem; | |
| 52 | } | |
| 53 | ||
| 54 | pre > code, time, code { | |
| 55 | font-family: Menlo, Consolas, Monaco, Liberation Mono, Lucida Console, monospace; | |
| 56 | font-size: 1rem; | |
| 57 | } | |
| 58 | ||
| 59 | pre { | |
| 60 | padding: 0.5rem; | |
| 61 | margin: 0.5rem 0; | |
| 62 | border: 1px solid #0d0d0d; | |
| 63 | overflow-x: auto; | |
| 64 | } | |
| 65 | ||
| 66 | :not(pre)>code { | |
| 67 | background-color: hsl(240, 100%, 85%); | |
| 68 | border: 1px solid | |
| 69 | hsl(240, 100%, 85%); | |
| 70 | border-radius: 0; | |
| 71 | color: hsl(0, 0%, 5%); | |
| 72 | padding: 0 | |
| 73 | 0.25rem; | |
| 74 | } | |
| 75 | ||
| 76 | aside { | |
| 77 | background-color: hsl(0, 0%, 85%); | |
| 78 | padding: 1rem; | |
| 79 | } | |
| 80 | ||
| 81 | blockquote { | |
| 82 | background-color: hsl(0, 0%, 85%); | |
| 83 | border-left: 5px solid hsl(240, 100%, 35%); | |
| 84 | color: hsl(0, 0%, 5%); | |
| 85 | padding: 1rem; | |
| 86 | margin: 1rem 0; | |
| 87 | } | |
| 88 | ||
| 89 | blockquote p { | |
| 90 | margin: 0; | |
| 91 | } | |
| 92 | ||
| 93 | .footnote-definition { | |
| 94 | display: flex; | |
| 95 | justify-content: flex-start; | |
| 96 | align-items: top; | |
| 97 | margin: 1rem 0; | |
| 98 | } | |
| 99 | ||
| 100 | .footnote-definition sup { | |
| 101 | font-size: inherit; | |
| 102 | top: 0; | |
| 103 | line-height: inherit; | |
| 104 | padding-right: .5rem; | |
| 105 | } | |
| 106 | ||
| 107 | .footnote-definition sup::after { | |
| 108 | content: "."; | |
| 109 | } | |
| 110 | ||
| 111 | .footnote-definition p { | |
| 112 | display: inline-block; | |
| 113 | margin: 0; | |
| 114 | } | |
| 115 | ||
| 116 | @media (prefers-color-scheme: dark) { | |
| 117 | body { | |
| 118 | background-color: hsl(0, 0%, 5%); | |
| 119 | color: hsl(0, 0%, 75%); | |
| 120 | } | |
| 121 | ||
| 122 | nav, footer { | |
| 123 | border-color: hsl(0, 0%, 75%); | |
| 124 | } | |
| 125 | ||
| 126 | h1, h2, h3, h4, h5, h6 { | |
| 127 | color: hsl(0, 0%, 85%); | |
| 128 | } | |
| 129 | ||
| 130 | a, a:visited { | |
| 131 | color: hsl(120, 100%, 45%); | |
| 132 | } | |
| 133 | ||
| 134 | a:hover { | |
| 135 | background-color: hsl(120, 100%, 45%); | |
| 136 | color: hsl(0, 0%, 5%); | |
| 137 | } | |
| 138 | :not(pre)>code { | |
| 139 | background-color: hsl(120, 100%, 35%); | |
| 140 | border: 1px solid hsl(120, 100%, 35%); | |
| 141 | border-radius: 0; | |
| 142 | color: hsl(0, 0%, 5%); | |
| 143 | padding: 0 0.25rem; | |
| 144 | } | |
| 145 | ||
| 146 | aside { | |
| 147 | background-color: hsl(0, 0%, 15%); | |
| 148 | padding: 1rem; | |
| 149 | } | |
| 150 | ||
| 151 | blockquote { | |
| 152 | background-color: hsl(0, 0%, 15%); | |
| 153 | border-left: 5px solid hsl(120, 100%, 25%); | |
| 154 | color: hsl(0, 0%, 85%); | |
| 155 | } | |
| 156 | ||
| 157 | pre { | |
| 158 | border: 1px solid hsl(0, 0%, 35%); | |
| 159 | } | |
| 160 | ||
| 161 | table { | |
| 162 | color: hsl(0, 0%, 75%); | |
| 163 | border-color: hsl(0, 0%, 35%); | |
| 164 | } | |
| 165 | ||
| 166 | thead, th, tr, td { | |
| 167 | border: 1px solid hsl(0, 0%, 35%); | |
| 168 | } | |
| 169 | } | |
wiki/index.org added +144
| @@ -0,0 +1,144 @@ | ||
| 1 | #+title: Wiki | |
| 2 | ||
| 3 | An informal wiki of sorts. | |
| 4 | ||
| 5 | * Digital Garden | |
| 6 | ||
| 7 | #+BEGIN_QUOTE | |
| 8 | At times, wilderness is exactly what readers want: a rich collection of | |
| 9 | resources and links. At times, rigid formality suits readers perfectly, | |
| 10 | providing precisely the information they want, no more and no less. Indeed, | |
| 11 | individual hypertexts and Web sites may contain sections that tend toward | |
| 12 | each extreme. | |
| 13 | #+END_QUOTE | |
| 14 | ||
| 15 | #+BEGIN_QUOTE | |
| 16 | Often, however, designers should strive for the comfort, interest, and | |
| 17 | habitability of parks and gardens: places that invite visitors to remain, and | |
| 18 | that are designed to engage and delight them, to invite them to linger, to | |
| 19 | explore, and to reflect. | |
| 20 | #+END_QUOTE | |
| 21 | ||
| 22 | [[https://www.eastgate.com/garden/][Hypertext Garden]] | |
| 23 | ||
| 24 | * Git | |
| 25 | ||
| 26 | I want to get rid of all local modifications and go back to the working tree | |
| 27 | of the most recent commit: | |
| 28 | ||
| 29 | #+BEGIN_SRC sh | |
| 30 | git restore . | |
| 31 | #+END_SRC | |
| 32 | ||
| 33 | Revert a specified commit: | |
| 34 | ||
| 35 | #+BEGIN_SRC sh | |
| 36 | git revert commit-id | |
| 37 | #+END_SRC | |
| 38 | ||
| 39 | Reset the repository to a specific commit in the git log: | |
| 40 | ||
| 41 | #+BEGIN_SRC sh | |
| 42 | git reset --mixed commit-id | |
| 43 | #+END_SRC | |
| 44 | ||
| 45 | I need to commit and push changes to a remote that has been changed since my | |
| 46 | most recent pull: | |
| 47 | ||
| 48 | #+BEGIN_SRC sh | |
| 49 | git pull --rebase | |
| 50 | #+END_SRC | |
| 51 | ||
| 52 | * Hardware | |
| 53 | ||
| 54 | ** Laptops | |
| 55 | ||
| 56 | *** macOS | |
| 57 | ||
| 58 | | Category | Details | | |
| 59 | | Model | Macbook Pro 16" | | |
| 60 | | CPU | Apple M2 Pro | | |
| 61 | | RAM | 16GB | | |
| 62 | | Storage | 512GB SSD | | |
| 63 | ||
| 64 | *** Linux | |
| 65 | ||
| 66 | | Category | Details | | |
| 67 | |----------|---------------------------------------------| | |
| 68 | | Model | Lenovo ThinkPad E15 Gen 4, model 21ED0048US | | |
| 69 | | CPU | AMD Ryzen 5 5625U with Radeon Graphics | | |
| 70 | | RAM | 16 GB | | |
| 71 | | Storage | 256 GB SSD | | |
| 72 | ||
| 73 | ** Servers | |
| 74 | ||
| 75 | | Category | Details | | |
| 76 | |--------------------|----------------------------------------| | |
| 77 | | Case | Rosewill RSV-R4100U 4U | | |
| 78 | | Motherboard | NZXT B550 | | |
| 79 | | CPU | AMD Ryzen 7 5700G with Radeon Graphics | | |
| 80 | | RAM | 64GB RAM (2x32GB) | | |
| 81 | | Storage (On-board) | Western Digital 500GB M.2 NVME SSD | | |
| 82 | | Storage (HDD Bay) | 48TB HDD | | |
| 83 | | PSU | Corsair RM850 PSU | | |
| 84 | ||
| 85 | ** Networking Equipment | |
| 86 | ||
| 87 | - UDM-Pro | |
| 88 | - USW-24-PoE | |
| 89 | - USW-Lite-8-PoE | |
| 90 | - U6-Pro | |
| 91 | - U6-Extender | |
| 92 | - USP-Plug | |
| 93 | - UVC G4 Instant x 3 | |
| 94 | - UVC G4 Doorbell Pro | |
| 95 | - UP Chime | |
| 96 | - USW 24-Port Patch Panel | |
| 97 | - USW Switch Lite 8 PoE | |
| 98 | ||
| 99 | * Software | |
| 100 | ||
| 101 | ** Laptop | |
| 102 | ||
| 103 | Alpine 3.18.2; no DE. | |
| 104 | ||
| 105 | I currently run my Alpine laptop via the default login shell - no desktop | |
| 106 | environment. From here, I use a mix of byobu and emacs to split my screen into | |
| 107 | tabs and panes. All programs run through the shell and do not use visual | |
| 108 | libraries such as X or Wayland. | |
| 109 | ||
| 110 | I have Sway installed and configured, but only launch it when I must. | |
| 111 | ||
| 112 | - brightnessctl | |
| 113 | - byobu | |
| 114 | - emacs | |
| 115 | - [font-dejavu, font-noto, font-noto-cjk, font-noto-cjk-extra] | |
| 116 | - glances | |
| 117 | - gnupg | |
| 118 | - irssi | |
| 119 | - lynx | |
| 120 | - nano | |
| 121 | - neomutt | |
| 122 | - newsboat | |
| 123 | - ohmyzsh | |
| 124 | - [pango, pango-tools] | |
| 125 | - pipewire | |
| 126 | - syncthing | |
| 127 | - wireguard | |
| 128 | - zola | |
| 129 | - zsh | |
| 130 | ||
| 131 | ** Server | |
| 132 | ||
| 133 | Ubuntu 22.04.1; no DE. | |
| 134 | ||
| 135 | See my services page for a list of the publicly-available services running on | |
| 136 | this server. | |
| 137 | ||
| 138 | - certbot | |
| 139 | - [docker, docker-compose] | |
| 140 | - nginx | |
| 141 | - zsh | |
| 142 | ||
| 143 | ||
| 144 | ||