Commit 972b15ecc9

972b15ecc9140bc41c0b80bcea7e71a4935cf4d3

parent: 7aef1980b5

Unsigned

cmc <hello@cleberg.net> · 2024-09-02 02:54 UTC

format 2018 to 2020 blog posts

Layout: unified · split

content/blog/2018-11-28-aes-encryption.org +79 −97
@@ -5,121 +5,103 @@
55
66* Basic AES
77
8If you're not familiar with encryption techniques,
9[[https://en.wikipedia.org/wiki/Advanced_Encryption_Standard][AES]] is
10the *Advanced Encryption Standard*. This specification was established
11by the National Institute of Standards and Technology, sub-selected from
12the Rijndael family of ciphers (128, 192, and 256 bits) in 2001.
13Furthering its popularity and status, the US government chose AES as
14their default encryption method for top-secret data, removing the
15previous standard which had been in place since 1977.
16
17AES has proven to be an extremely safe encryption method, with 7-round
18and 8-round attacks making no material improvements since the release of
19this encryption standard almost two decades ago.
8If you're not familiar with encryption techniques, [[https://en.wikipedia.org/wiki/Advanced_Encryption_Standard][AES]] is the *Advanced
9Encryption Standard*. This specification was established by the National
10Institute of Standards and Technology, sub-selected from the Rijndael family of
11ciphers (128, 192, and 256 bits) in 2001. Furthering its popularity and status,
12the US government chose AES as their default encryption method for top-secret
13data, removing the previous standard which had been in place since 1977.
14
15AES has proven to be an extremely safe encryption method, with 7-round and
168-round attacks making no material improvements since the release of this
17encryption standard almost two decades ago.
2018
2119#+begin_quote
22Though many papers have been published on the cryptanalysis of AES, the
23fastest single-key attacks on round-reduced AES variants [20, 33] so far
24are only slightly more powerful than those proposed 10 years ago
25[23,24].
20Though many papers have been published on the cryptanalysis of AES, the fastest
21single-key attacks on round-reduced AES variants [20, 33] so far are only
22slightly more powerful than those proposed 10 years ago [23,24].
2623
27- [[http://research.microsoft.com/en-us/projects/cryptanalysis/aesbc.pdf][Bogdonav,
28 et al.]]
24- [[http://research.microsoft.com/en-us/projects/cryptanalysis/aesbc.pdf][Bogdonav, et al.]]
2925#+end_quote
3026
3127* How Secure is AES?
3228
33In theory, AES-256 is non-crackable due to the massive number of
34combinations that can be produced. However, AES-128 is no longer
35recommended as a viable implementation to protect important data.
36
37A semi-short
38[[http://www.moserware.com/2009/09/stick-figure-guide-to-advanced.html][comic
39strip]] from Moserware quickly explains AES for the public to
40understand. Basically AES encrypts the data by obscuring the
41relationship between the data and the encrypted data. Additionally, this
42method spreads the message out. Lastly, the key produced by AES is the
43secret to decrypting it. Someone may know the method of AES, but without
44the key, they are powerless.
45
46To obscure and spread the data out, AES creates a
47substitution-permutation network. Wikipedia has a wonderful
48[[https://upload.wikimedia.org/wikipedia/commons/thumb/c/cd/SubstitutionPermutationNetwork2.png/468px-SubstitutionPermutationNetwork2.png][example
49of an SP network]] available. This network sends the data through a set
50of S boxes (using the unique key) to substitute the bits with another
51block of bits. Then, a P box will permutate, or rearrange, the bits.
52This is done over and over, with the key being derived from the last
53round. For AES, the key size specifies the number of transformation
29In theory, AES-256 is non-crackable due to the massive number of combinations
30that can be produced. However, AES-128 is no longer recommended as a viable
31implementation to protect important data.
32
33A semi-short [[http://www.moserware.com/2009/09/stick-figure-guide-to-advanced.html][comic strip]] from Moserware quickly explains AES for the public to
34understand. Basically AES encrypts the data by obscuring the relationship
35between the data and the encrypted data. Additionally, this method spreads the
36message out. Lastly, the key produced by AES is the secret to decrypting it.
37Someone may know the method of AES, but without the key, they are powerless.
38
39To obscure and spread the data out, AES creates a substitution-permutation
40network. Wikipedia has a wonderful [[https://upload.wikimedia.org/wikipedia/commons/thumb/c/cd/SubstitutionPermutationNetwork2.png/468px-SubstitutionPermutationNetwork2.png][example of an SP network]] available. This
41network sends the data through a set of S boxes (using the unique key) to
42substitute the bits with another block of bits. Then, a P box will permutate, or
43rearrange, the bits. This is done over and over, with the key being derived from
44the last round. For AES, the key size specifies the number of transformation
5445rounds: 10, 12, and 14 rounds for 128-bit, 192-bit, and 256-bit keys,
5546respectively.
5647
5748* The Process
5849
591. *KeyExpansion*: Using
60 [[https://en.m.wikipedia.org/wiki/Advanced_Encryption_Standard][Rijndael's
61 key schedule]], the keys are dynamically generated.
622. *AddRoundKey*: Each byte of the data is combined with this key using
63 bitwise xor.
643. *SubBytes*: This is followed by the substitution of each byte of
65 data.
664. *ShiftRows*: Then, the final three rows are shifted a certain number
67 of steps, dictated by the cipher.
685. *MixColumns*: After the rows have been shifted, the columns are mixed
69 and combined.
70
71This process does not necessarily stop after one full round. Steps 2
72through 5 will repeat for the number of rounds specified by the key.
73However, the final round excludes the MixColumns step. As you can see,
74this is a fairly complex process. One must have a solid understanding of
75general mathematic principles to fully understand how the sequence works
76(and to even attempt to find a weakness).
77
78According to research done by Bogdanov et al., it would take billions of
79years to brute force a 126-bit key with current hardware. Additionally,
80this brute force attack would require storing 2^{88} bits of data!
81However, there are a few different attacks that have been used to show
82vulnerabilities with the use of this technology. Side-channel attacks
83use inadvertent leaks of data from the hardware or software, which can
84allow attackers to obtain the key or run programs on a user's hardware.
85
86Please note that this is not something you should run out and try to
87implement in your =Hello, World!= app after only a few hours of
88research. While AES (basically all encryption methods) is extremely
89efficient in what it does, it takes a lot of time and patience to
90understand. If you're looking for something which currently implements
91AES, check out the
92[[https://www.bouncycastle.org/documentation.html][Legion of the Bouncy
93Castle]] for Java implementations of cryptographic algorithms.
501. *KeyExpansion*: Using [[https://en.m.wikipedia.org/wiki/Advanced_Encryption_Standard][Rijndael's key schedule]], the keys are dynamically
51 generated.
522. *AddRoundKey*: Each byte of the data is combined with this key using bitwise
53 xor.
543. *SubBytes*: This is followed by the substitution of each byte of data.
554. *ShiftRows*: Then, the final three rows are shifted a certain number of
56 steps, dictated by the cipher.
575. *MixColumns*: After the rows have been shifted, the columns are mixed and
58 combined.
59
60This process does not necessarily stop after one full round. Steps 2 through 5
61will repeat for the number of rounds specified by the key. However, the final
62round excludes the MixColumns step. As you can see, this is a fairly complex
63process. One must have a solid understanding of general mathematic principles to
64fully understand how the sequence works (and to even attempt to find a
65weakness).
66
67According to research done by Bogdanov et al., it would take billions of years
68to brute force a 126-bit key with current hardware. Additionally, this brute
69force attack would require storing 2^{88} bits of data! However, there are a few
70different attacks that have been used to show vulnerabilities with the use of
71this technology. Side-channel attacks use inadvertent leaks of data from the
72hardware or software, which can allow attackers to obtain the key or run
73programs on a user's hardware.
74
75Please note that this is not something you should run out and try to implement
76in your =Hello, World!= app after only a few hours of research. While AES
77(basically all encryption methods) is extremely efficient in what it does, it
78takes a lot of time and patience to understand. If you're looking for something
79which currently implements AES, check out the [[https://www.bouncycastle.org/documentation.html][Legion of the Bouncy Castle]] for
80Java implementations of cryptographic algorithms.
9481
9582* Why Does Encryption Matter?
9683
97There are limitless reasons to enable encryption at-rest or in-transit
98for various aspects of your digital life. You can research specific
99examples, such as
100[[https://arstechnica.com/tech-policy/2018/12/australia-passes-new-law-to-thwart-strong-encryption/][Australia
101passes new law to thwart strong encryption]]. However, I will simply
84There are limitless reasons to enable encryption at-rest or in-transit for
85various aspects of your digital life. You can research specific examples, such
86as [[https://arstechnica.com/tech-policy/2018/12/australia-passes-new-law-to-thwart-strong-encryption/][Australia passes new law to thwart strong encryption]]. However, I will simply
10287list a few basic reasons to always enable encryption, where feasible:
10388
1041. Privacy is a human right and is recognized as a national right in
105 some countries (e.g.,
106 [[https://www.law.cornell.edu/wex/fourth_amendment][US Fourth
107 Amendment]]).
1082. "Why not?" Encryption rarely affects performance or speed, so there's
109 usually not a reason to avoid it in the first place.
1103. Your digital identity and activity (texts, emails, phone calls,
111 online accounts, etc.) are extremely valuable and can result in
112 terrible consequences, such as identity theft, if leaked to other
113 parties. Encrypting this data prevents such leaks from ruining lives.
1144. Wiping or factory-resetting does not actually wipe all data from the
115 storage device. There are methods to read data from the physical
116 disks/boards inside devices.
891. Privacy is a human right and is recognized as a national right in some
90 countries (e.g., [[https://www.law.cornell.edu/wex/fourth_amendment][US Fourth Amendment]]).
912. "Why not?" Encryption rarely affects performance or speed, so there's usually
92 not a reason to avoid it in the first place.
933. Your digital identity and activity (texts, emails, phone calls, online
94 accounts, etc.) are extremely valuable and can result in terrible
95 consequences, such as identity theft, if leaked to other parties. Encrypting
96 this data prevents such leaks from ruining lives.
974. Wiping or factory-resetting does not actually wipe all data from the storage
98 device. There are methods to read data from the physical disks/boards inside
99 devices.
1171005. Corporations, governments, and other nefarious groups/individuals are
118 actively looking for ways to collect personal information about
119 anyone they can. If someone's data is unencrypted, that person may
120 become a target due to the ease of data collection.
101 actively looking for ways to collect personal information about anyone they
102 can. If someone's data is unencrypted, that person may become a target due to
103 the ease of data collection.
121104
122105​*Read More:*
123106
124- [[http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.197.pdf][Federal
125 Information Processing Standards Publication 197]]
107- [[http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.197.pdf][Federal Information Processing Standards Publication 197]]
content/blog/2018-11-28-cpp-compiler.org +46 −53
@@ -5,14 +5,13 @@
55
66* A Brief Introduction
77
8[[https://en.wikipedia.org/wiki/C%2B%2B][C++]] is a general-purpose
9programming language with object-oriented, generic, and functional
10features in addition to facilities for low-level memory manipulation.
8[[https://en.wikipedia.org/wiki/C%2B%2B][C++]] is a general-purpose programming language with object-oriented, generic, and
9functional features in addition to facilities for low-level memory manipulation.
1110
12The source code, shown in the snippet below, must be compiled before it
13can be executed. There are many steps and intricacies to the compilation
14process, and this post was a personal exercise to learn and remember as
15much information as I can.
11The source code, shown in the snippet below, must be compiled before it can be
12executed. There are many steps and intricacies to the compilation process, and
13this post was a personal exercise to learn and remember as much information as I
14can.
1615
1716#+begin_src cpp
1817#include <iostream>
@@ -27,24 +26,19 @@ int main()
2726
2827*** An Overview
2928
30Compiling C++ projects is a frustrating task most days. Seemingly
31nonexistent errors keeping your program from successfully compiling can
32be annoying (especially since you know you wrote it perfectly the first
33time, right?).
34
35I'm learning more and more about C++ these days and decided to write
36this concept down so that I can cement it even further in my own head.
37However, C++ is not the only compiled language. Check out
38[[https://en.wikipedia.org/wiki/Compiled_language][the Wikipedia entry
39for compiled languages]] for more examples of compiled languages.
40
41I'll start with a wonderful, graphical way to conceptualize the C++
42compiler. View
43[[https://web.archive.org/web/20190419035048/http://faculty.cs.niu.edu/~mcmahon/CS241/Notes/compile.html][The
44C++ Compilation Process]] by Kurt MacMahon, an NIU professor, to see the
45graphic and an explanation. The goal of the compilation process is to
46take the C++ code and produce a shared library, dynamic library, or an
47executable file.
29Compiling C++ projects is a frustrating task most days. Seemingly nonexistent
30errors keeping your program from successfully compiling can be annoying
31(especially since you know you wrote it perfectly the first time, right?).
32
33I'm learning more and more about C++ these days and decided to write this
34concept down so that I can cement it even further in my own head. However, C++
35is not the only compiled language. Check out [[https://en.wikipedia.org/wiki/Compiled_language][the Wikipedia entry for compiled
36languages]] for more examples of compiled languages.
37
38I'll start with a wonderful, graphical way to conceptualize the C++ compiler.
39View [[https://web.archive.org/web/20190419035048/http://faculty.cs.niu.edu/~mcmahon/CS241/Notes/compile.html][The C++ Compilation Process]] by Kurt MacMahon, an NIU professor, to see the
40graphic and an explanation. The goal of the compilation process is to take the
41C++ code and produce a shared library, dynamic library, or an executable file.
4842
4943** Compilation Phases
5044
@@ -53,28 +47,27 @@ compiling C++ code.
5347
5448*** Step 1
5549
56The first step is to expand the source code file to meet all
57dependencies. The C++ preprocessor includes the code from all the header
58files, such as =#include <iostream>=. Now, what does that mean? The
59previous example includes the =iostream= header. This tells the computer
60that you want to use the =iostream= standard library, which contains
61classes and functions written in the core language. This specific header
62allows you to manipulate input/output streams. After all this, you'll
63end up which a temporary file that contains the expanded source code.
50The first step is to expand the source code file to meet all dependencies. The
51C++ preprocessor includes the code from all the header files, such as =#include
52<iostream>=. Now, what does that mean? The previous example includes the
53=iostream= header. This tells the computer that you want to use the =iostream=
54standard library, which contains classes and functions written in the core
55language. This specific header allows you to manipulate input/output streams.
56After all this, you'll end up which a temporary file that contains the expanded
57source code.
6458
65In the example of the C++ code above, the =iostream= class would be
66included in the expanded code.
59In the example of the C++ code above, the =iostream= class would be included in
60the expanded code.
6761
6862*** Step 2
6963
70After the code is expanded, the compiler comes into play. The compiler
71takes the C++ code and converts this code into the assembly language,
72understood by the platform. You can see this in action if you head over
73to the [[https://godbolt.org][GodBolt Compiler Explorer]], which shows
74C++ being converted into assembly dynamically.
64After the code is expanded, the compiler comes into play. The compiler takes the
65C++ code and converts this code into the assembly language, understood by the
66platform. You can see this in action if you head over to the [[https://godbolt.org][GodBolt Compiler
67Explorer]], which shows C++ being converted into assembly dynamically.
7568
76For example, the =Hello, world!= code snippet above compiles into the
77following assembly code:
69For example, the =Hello, world!= code snippet above compiles into the following
70assembly code:
7871
7972#+begin_src asm
8073.LC0:
@@ -120,17 +113,17 @@ _GLOBAL__sub_I_main:
120113
121114*** Step 3
122115
123Third, the assembly code generated by the compiler is assembled into the
124object code for the platform. Essentially, this is when the compiler
125takes the assembly code and assembles it into machine code in a binary
126format. After researching this online, I figured out that a lot of
127compilers will allow you to stop compilation at this step. This would be
128useful for compiling each source code file separately. This saves time
129later if a single file changes; only that file needs to be recompiled.
116Third, the assembly code generated by the compiler is assembled into the object
117code for the platform. Essentially, this is when the compiler takes the assembly
118code and assembles it into machine code in a binary format. After researching
119this online, I figured out that a lot of compilers will allow you to stop
120compilation at this step. This would be useful for compiling each source code
121file separately. This saves time later if a single file changes; only that file
122needs to be recompiled.
130123
131124*** Step 4
132125
133Finally, the object code file generated by the assembler is linked
134together with the object code files for any library functions used to
135produce a shared library, dynamic library, or an executable file. It
136replaces all references to undefined symbols with the correct addresses.
126Finally, the object code file generated by the assembler is linked together with
127the object code files for any library functions used to produce a shared
128library, dynamic library, or an executable file. It replaces all references to
129undefined symbols with the correct addresses.
content/blog/2019-01-07-useful-css.org +50 −61
@@ -5,35 +5,31 @@
55
66* Introduction to CSS
77
8[[https://en.wikipedia.org/wiki/CSS][CSS]], the language used to markup
9HTML code and make it "pretty", is one of the most effective ways to
10increase the attractiveness of a website. It can also lead to increased
11user engagement, retention, and satisfaction. In fact, there are whole
12career fields are dedicated to the improvement of user experiences,
8[[https://en.wikipedia.org/wiki/CSS][CSS]], the language used to markup HTML code and make it "pretty", is one of the
9most effective ways to increase the attractiveness of a website. It can also
10lead to increased user engagement, retention, and satisfaction. In fact, there
11are whole career fields are dedicated to the improvement of user experiences,
1312known as UI design and UX design.
1413
15Some web developers are used to the common CSS properties, such as
16element sizing, fonts, colors, etc., but are not as well versed in
17less-used properties and values such as =flexbox=, =clip-path=, and
18=transform=. This article will provide some insight into the less-used
19and unique CSS properties.
14Some web developers are used to the common CSS properties, such as element
15sizing, fonts, colors, etc., but are not as well versed in less-used properties
16and values such as =flexbox=, =clip-path=, and =transform=. This article will
17provide some insight into the less-used and unique CSS properties.
2018
2119* CSS Variables
2220
23The first topic today is CSS variables. Variables are not often used by
24smaller developers. CSS variables allow you to give your website a
25well-defined structure, where you can easily reuse CSS properties
26throughout the project.
21The first topic today is CSS variables. Variables are not often used by smaller
22developers. CSS variables allow you to give your website a well-defined
23structure, where you can easily reuse CSS properties throughout the project.
2724
28You can use variables to define things, such as color palettes. Then,
29you can use these colors for backgrounds anywhere else in the HTML. This
30could be extended, where extra variables could be defined for
31=primary-text=, =quoted-text=, etc. Variables can also be used to define
32spacing (e.g. =32px= or =2rem=), which can then be applied to margins,
33padding, font sizes, and more.
25You can use variables to define things, such as color palettes. Then, you can
26use these colors for backgrounds anywhere else in the HTML. This could be
27extended, where extra variables could be defined for =primary-text=,
28=quoted-text=, etc. Variables can also be used to define spacing (e.g. =32px= or
29=2rem=), which can then be applied to margins, padding, font sizes, and more.
3430
35For example, here are some variables defined at the root of the website,
36which allows for any subsequent CSS rules to use those variables:
31For example, here are some variables defined at the root of the website, which
32allows for any subsequent CSS rules to use those variables:
3733
3834#+begin_src css
3935:root {
@@ -49,23 +45,21 @@ body {
4945
5046* CSS Box Shadows
5147
52Box shadows were once my mortal enemy. No matter how hard I tried, I
53just couldn't get them to work how I wanted. Because of this, my
54favorite discovery has been CSSMatic's
55[[https://www.cssmatic.com/box-shadow][box shadow generator]]. It
56provides an excellent tool to generate box shadows using their simple
57sliders. Surprisingly, this is the reason I learned how box shadows
58work! You can use the sliders and watch how the CSS code changes in the
59image that is displayed. Through this, you should understand that the
60basic structure for box shadows is:
48Box shadows were once my mortal enemy. No matter how hard I tried, I just
49couldn't get them to work how I wanted. Because of this, my favorite discovery
50has been CSSMatic's [[https://www.cssmatic.com/box-shadow][box shadow generator]]. It provides an excellent tool to
51generate box shadows using their simple sliders. Surprisingly, this is the
52reason I learned how box shadows work! You can use the sliders and watch how the
53CSS code changes in the image that is displayed. Through this, you should
54understand that the basic structure for box shadows is:
6155
6256#+begin_src css
6357box-shadow: inset horizontal vertical blur spread color;
6458#+end_src
6559
66Now, let's look at some basic examples! You can copy and paste the
67following code into a site like CodePen or your own HTML files. Feel
68free to play around with the code, experiment, and learn.
60Now, let's look at some basic examples! You can copy and paste the following
61code into a site like CodePen or your own HTML files. Feel free to play around
62with the code, experiment, and learn.
6963
7064*Box Shadow #1*
7165
@@ -121,22 +115,20 @@ free to play around with the code, experiment, and learn.
121115}
122116#+end_src
123117
124Try these box shadows out on your own and see how changing each shadow
125value works.
118Try these box shadows out on your own and see how changing each shadow value
119works.
126120
127121* CSS Flexbox
128122
129Now, let's move on to the best part of this article: flexbox. The
130flexbox is by far my favorite new toy. I originally stumbled across this
131solution after looking for more efficient ways of centering content
132horizontally AND vertically. I had used a few hack-ish methods before,
133but flexbox throws those out the window. The best part of it all is that
134flexbox is /dead simple/.
123Now, let's move on to the best part of this article: flexbox. The flexbox is by
124far my favorite new toy. I originally stumbled across this solution after
125looking for more efficient ways of centering content horizontally AND
126vertically. I had used a few hack-ish methods before, but flexbox throws those
127out the window. The best part of it all is that flexbox is /dead simple/.
135128
136Flexbox pertains to the parent div of any element. You want the parent
137to be the flexbox in which items are arranged to use the flex methods.
138It's easier to see this in action that explained, so let's see an
139example.
129Flexbox pertains to the parent div of any element. You want the parent to be the
130flexbox in which items are arranged to use the flex methods. It's easier to see
131this in action that explained, so let's see an example.
140132
141133*Flexbox*
142134
@@ -171,24 +163,21 @@ example.
171163}
172164#+end_src
173165
174You may notice that we no longer need to use the =top= property for the
175=h3= elements in our code. This is because we set the display box to be
176a flex container for the small boxes, AND we made the small boxes flex
177containers for their elements (the h3 tags). Flex boxes can be nested
178like this to center content that is inside centered content.
166You may notice that we no longer need to use the =top= property for the =h3=
167elements in our code. This is because we set the display box to be a flex
168container for the small boxes, AND we made the small boxes flex containers for
169their elements (the h3 tags). Flex boxes can be nested like this to center
170content that is inside centered content.
179171
180For the example above, we designated the =justify-content= property to
181be =flex-start= so that the boxes stack from the left side of the
182screen. This property can be changed to =center= to make the boxes
183appear in the center of the screen.
172For the example above, we designated the =justify-content= property to be
173=flex-start= so that the boxes stack from the left side of the screen. This
174property can be changed to =center= to make the boxes appear in the center of
175the screen.
184176
185For an interactive example,
186[[https://codepen.io/LandonSchropp/pen/KpzzGo][check out this CodePen]]
187from [[https://codepen.io/LandonSchropp/][LandonScropp]]. Resize the
177For an interactive example, [[https://codepen.io/LandonSchropp/pen/KpzzGo][check out this CodePen]] from [[https://codepen.io/LandonSchropp/][LandonScropp]]. Resize the
188178window with dice to see how they collapse and re-align.
189179
190180* Even More CSS
191181
192For more inspiration, you can visit [[https://www.codepen.io][CodePen]],
193[[https://dribbble.com][Dribbble]], or [[https://uimovement.com][UI
194Movement]] to browse the collections of many amazing web designers.
182For more inspiration, you can visit [[https://www.codepen.io][CodePen]], [[https://dribbble.com][Dribbble]], or [[https://uimovement.com][UI Movement]] to browse
183the collections of many amazing web designers.
content/blog/2019-09-09-audit-analytics.org +142 −160
@@ -5,107 +5,99 @@
55
66* What Are Data Analytics?
77
8A quick aside before I dive into this post: =data analytics= is a vague
9term that has become popular in recent years. Think of a =data analytic=
10as the output of any data analysis you perform. For example, a pivot
11table or a pie chart could be a data analytic.
12
13[[https://en.wikipedia.org/wiki/Data_analysis][Data analysis]] is a
14process that utilizes statistics and other mathematical methods to
15discover useful information within datasets. This involves examining,
16cleaning, transforming, and modeling data so that you can use the data
17to support an opinion, create more useful viewpoints, and gain knowledge
18to implement into audit planning or risk assessments.
19
20One of the common mistakes that managers (and anyone new to the process)
21make is assuming that everything involved with this process is "data
22analytics". In fact, data analytics are only a small part of the
23process.
24
25See *Figure 1* for a more accurate representation of where data analysis
26sits within the full process. This means that data analysis does not
27include querying or extracting data, selecting samples, or performing
28audit tests. These steps can be necessary for an audit (and may even be
29performed by the same associates), but they are not data analytics.
8A quick aside before I dive into this post: =data analytics= is a vague term
9that has become popular in recent years. Think of a =data analytic= as the
10output of any data analysis you perform. For example, a pivot table or a pie
11chart could be a data analytic.
12
13[[https://en.wikipedia.org/wiki/Data_analysis][Data analysis]] is a process that utilizes statistics and other mathematical
14methods to discover useful information within datasets. This involves examining,
15cleaning, transforming, and modeling data so that you can use the data to
16support an opinion, create more useful viewpoints, and gain knowledge to
17implement into audit planning or risk assessments.
18
19One of the common mistakes that managers (and anyone new to the process) make is
20assuming that everything involved with this process is "data analytics". In
21fact, data analytics are only a small part of the process.
22
23See *Figure 1* for a more accurate representation of where data analysis sits
24within the full process. This means that data analysis does not include querying
25or extracting data, selecting samples, or performing audit tests. These steps
26can be necessary for an audit (and may even be performed by the same
27associates), but they are not data analytics.
3028
3129* Current Use of Analytics in Auditing
3230
33While data analysis has been an integral part of most businesses and
34departments for the better part of the last century, only recently have
35internal audit functions been adopting this practice. The internal audit
36function works exclusively to provide assurance and consulting services
37to the business areas within the firm (except for internal auditing
38firms who are hired by different companies to perform their roles).
31While data analysis has been an integral part of most businesses and departments
32for the better part of the last century, only recently have internal audit
33functions been adopting this practice. The internal audit function works
34exclusively to provide assurance and consulting services to the business areas
35within the firm (except for internal auditing firms who are hired by different
36companies to perform their roles).
3937
4038#+begin_quote
41Internal Auditing helps an organization accomplish its objectives by
42bringing a systematic, disciplined approach to evaluate and improve the
43effectiveness of risk management, control and governance processes.
39Internal Auditing helps an organization accomplish its objectives by bringing a
40systematic, disciplined approach to evaluate and improve the effectiveness of
41risk management, control and governance processes.
4442
4543- The IIA's Definition of Internal Audit
4644#+end_quote
4745
48Part of the blame for the slow adoption of data analysis can be
49attributed to the fact that internal auditing is strongly based on
50tradition and following the precedents set by previous auditors.
51However, there can be no progress without auditors who are willing to
52break the mold and test new audit techniques. In fact, as of 2018,
53[[https://www.cpapracticeadvisor.com/accounting-audit/news/12404086/internal-audit-groups-are-lagging-in-data-analytics][only
5463% of internal audit departments currently utilize data analytics]] in
55North America. This number should be as close as possible to 100%. I
56have never been part of an audit that would not have benefited from data
46Part of the blame for the slow adoption of data analysis can be attributed to
47the fact that internal auditing is strongly based on tradition and following the
48precedents set by previous auditors. However, there can be no progress without
49auditors who are willing to break the mold and test new audit techniques. In
50fact, as of 2018, [[https://www.cpapracticeadvisor.com/accounting-audit/news/12404086/internal-audit-groups-are-lagging-in-data-analytics][only 63% of internal audit departments currently utilize data
51analytics]] in North America. This number should be as close as possible to 100%.
52I have never been part of an audit that would not have benefited from data
5753analytics.
5854
59So, how do internal audit functions remedy this situation? It's
60definitely not as easy as walking into work on Monday and telling your
61Chief Audit Executive that you're going to start implementing analytics
62in the next audit. You need a plan and a system to make the analysis
63process as effective as possible.
55So, how do internal audit functions remedy this situation? It's definitely not
56as easy as walking into work on Monday and telling your Chief Audit Executive
57that you're going to start implementing analytics in the next audit. You need a
58plan and a system to make the analysis process as effective as possible.
6459
6560* The DELTA Model
6661
6762One of the easiest ways to experiment with data analytics and gain an
68understanding of the processes is to implement them within your own
69department. But how do we do this if we've never worked with analysis
70before? One of the most common places to start is to research some data
71analysis models currently available. For this post, we'll take a look at
72the DELTA model. You can take a look at *Figure 2* for a quick overview
73of the model.
63understanding of the processes is to implement them within your own department.
64But how do we do this if we've never worked with analysis before? One of the
65most common places to start is to research some data analysis models currently
66available. For this post, we'll take a look at the DELTA model. You can take a
67look at *Figure 2* for a quick overview of the model.
7468
75The DELTA model sets a few guidelines for areas wanting to implement
76data analytics so that the results can be as comprehensive as possible:
69The DELTA model sets a few guidelines for areas wanting to implement data
70analytics so that the results can be as comprehensive as possible:
7771
7872- *Data*: Must be clean, accessible, and (usually) unique.
79- *Enterprise-Wide Focus*: Key data systems and analytical resources
80 must be available for use (by the Internal Audit Function).
81- *Leaders*: Must promote a data analytics approach and show the value
82 of analytical results.
83- *Targets*: Must be set for key areas and risks that the analytics can
84 be compared against (KPIs).
85- *Analysts*: There must be auditors willing and able to perform data
86 analytics or else the system cannot be sustained.
73- *Enterprise-Wide Focus*: Key data systems and analytical resources must be
74 available for use (by the Internal Audit Function).
75- *Leaders*: Must promote a data analytics approach and show the value of
76 analytical results.
77- *Targets*: Must be set for key areas and risks that the analytics can be
78 compared against (KPIs).
79- *Analysts*: There must be auditors willing and able to perform data analytics
80 or else the system cannot be sustained.
8781
8882* Finding the Proper KPIs
8983
90Once the Internal Audit Function has decided that they want to start
91using data analytics internally and have ensured they're properly set up
92to do so, they need to figure out what they will be testing against. Key
93Performance Indicators (KPIs) are qualitative or quantitative factors
94that can be evaluated and assessed to determine if the department is
95performing well, usually compared to historical or industry benchmarks.
96Once KPIs have been agreed upon and set, auditors can use data analytics
97to assess and report on these KPIs. This allows the person performing
98the analytics the freedom to express opinions on the results, whereas
99the results are ambiguous if no KPIs exist.
100
101It should be noted that tracking KPIs in the department can help ensure
102you have a rigorous Quality Assurance and Improvement Program (QAIP) in
103accordance with some applicable standards, such as IPPF Standard 1300.
84Once the Internal Audit Function has decided that they want to start using data
85analytics internally and have ensured they're properly set up to do so, they
86need to figure out what they will be testing against. Key Performance Indicators
87(KPIs) are qualitative or quantitative factors that can be evaluated and
88assessed to determine if the department is performing well, usually compared to
89historical or industry benchmarks. Once KPIs have been agreed upon and set,
90auditors can use data analytics to assess and report on these KPIs. This allows
91the person performing the analytics the freedom to express opinions on the
92results, whereas the results are ambiguous if no KPIs exist.
93
94It should be noted that tracking KPIs in the department can help ensure you have
95a rigorous Quality Assurance and Improvement Program (QAIP) in accordance with
96some applicable standards, such as IPPF Standard 1300.
10497
10598#+begin_quote
106The chief audit executive must develop and maintain a quality assurance
107and improvement program that covers all aspects of the internal audit
108activity.
99The chief audit executive must develop and maintain a quality assurance and
100improvement program that covers all aspects of the internal audit activity.
109101
110102- IPPF Standard 1300
111103#+end_quote
@@ -113,38 +105,36 @@ activity.
113105Additionally, IPPF Standard 2060 discusses reporting:
114106
115107#+begin_quote
116The chief audit executive must report periodically to senior management
117and the board on the internal audit activity's purpose, authority,
118responsibility, and performance relative to its plan and on its
119conformance with the Code of Ethics and the Standards. Reporting must
120also include significant risk and control issues, including fraud risks,
121governance issues, and other matters that require the attention of
122senior management and/or the board.
108The chief audit executive must report periodically to senior management and the
109board on the internal audit activity's purpose, authority, responsibility, and
110performance relative to its plan and on its conformance with the Code of Ethics
111and the Standards. Reporting must also include significant risk and control
112issues, including fraud risks, governance issues, and other matters that require
113the attention of senior management and/or the board.
123114
124115- IPPF Standard 2060
125116#+end_quote
126117
127The hardest part of finding KPIs is to determine which KPIs are
128appropriate for your department. Since every department is different and
129has different goals, KPIs will vary drastically between companies. To
130give you an idea of where to look, here are some ideas I came up with
131when discussing the topic with a few colleagues.
118The hardest part of finding KPIs is to determine which KPIs are appropriate for
119your department. Since every department is different and has different goals,
120KPIs will vary drastically between companies. To give you an idea of where to
121look, here are some ideas I came up with when discussing the topic with a few
122colleagues.
132123
133124- Efficiency/Budgeting:
134 - Audit hours to staff utilization ratio (annual hours divided by
135 total annual work hours).
125 - Audit hours to staff utilization ratio (annual hours divided by total annual
126 work hours).
136127 - Audit hours compared to the number of audits completed.
137 - Time between audit steps or to complete the whole audit. E.g., time
138 from fieldwork completion to audit report issuance.
128 - Time between audit steps or to complete the whole audit. E.g., time from
129 fieldwork completion to audit report issuance.
139130- Reputation:
140131 - The frequency that management has requested the services of the IAF.
141 - Management, audit committee, or external audit satisfaction survey
142 results.
143 - Education, experience, certifications, tenure, and training of the
144 auditors on staff.
132 - Management, audit committee, or external audit satisfaction survey results.
133 - Education, experience, certifications, tenure, and training of the auditors
134 on staff.
145135- Quality:
146 - Number and frequency of audit findings. Assign monetary or numerical
147 values, if possible.
136 - Number and frequency of audit findings. Assign monetary or numerical values,
137 if possible.
148138 - Percentage of recommendations issued and implemented.
149139- Planning:
150140 - Percentage or number of key risks audited per year or per audit.
@@ -152,69 +142,61 @@ when discussing the topic with a few colleagues.
152142
153143* Data Analysis Tools
154144
155Finally, to be able to analyze and report on the data analysis, auditors
156need to evaluate the tools at their disposal. There are many options
157available, but a few of the most common ones can easily get the job
158done. For example, almost every auditor already has access to Microsoft
159Excel. Excel is more powerful than most people give it credit for and
160can accomplish a lot of basic statistics without much work. If you don't
161know a lot about statistics but still want to see some of the more basic
162results, Excel is a great option.
163
164To perform more in-depth statistical analysis or to explore large
165datasets that Excel cannot handle, auditors will need to explore other
166options. The big three that have had a lot of success in recent years
167are Python, R, and ACL. ACL can be used as either a graphical tool
168(point and click) or as a scripting tool, where the auditor must write
169the scripts manually. Python and the R-language are solely scripting
170languages.
171
172The general trend in the data analytics environment is that if the tool
173allows you to do everything by clicking buttons or dragging elements,
174you won't be able to fully utilize the analytics you need. The most
175robust solutions are created by those who understand how to write the
176scripts manually. It should be noted that as the utility of a tool
177increases, it usually means that the learning curve for that tool will
178also be higher. It will take auditors longer to learn how to utilize
179Python, R, or ACL versus learning how to utilize Excel.
145Finally, to be able to analyze and report on the data analysis, auditors need to
146evaluate the tools at their disposal. There are many options available, but a
147few of the most common ones can easily get the job done. For example, almost
148every auditor already has access to Microsoft Excel. Excel is more powerful than
149most people give it credit for and can accomplish a lot of basic statistics
150without much work. If you don't know a lot about statistics but still want to
151see some of the more basic results, Excel is a great option.
152
153To perform more in-depth statistical analysis or to explore large datasets that
154Excel cannot handle, auditors will need to explore other options. The big three
155that have had a lot of success in recent years are Python, R, and ACL. ACL can
156be used as either a graphical tool (point and click) or as a scripting tool,
157where the auditor must write the scripts manually. Python and the R-language are
158solely scripting languages.
159
160The general trend in the data analytics environment is that if the tool allows
161you to do everything by clicking buttons or dragging elements, you won't be able
162to fully utilize the analytics you need. The most robust solutions are created
163by those who understand how to write the scripts manually. It should be noted
164that as the utility of a tool increases, it usually means that the learning
165curve for that tool will also be higher. It will take auditors longer to learn
166how to utilize Python, R, or ACL versus learning how to utilize Excel.
180167
181168* Visualization
182169
183Once an auditor has finally found the right data, KPIs, and tools, they
184must report these results so that actions can be taken. Performing
185in-depth data analysis is only useful if the results are understood by
186the audiences of the data. The best way to create this understanding is
187to visualize the results of the data. Let's take a look at some of the
188best options to visualize and report the results you've found.
189
190Some of the most popular commercial tools for visualization are
191Microsoft PowerBI and Tableau Desktop. However, other tools exist such
192as JMP, Plotly, Qlikview, Alteryx, or D3. Some require commercial
193licenses while others are simply free to use. For corporate data, you
194may want to make sure that the tool does not communicate any of the data
195outside the company (such as cloud storage). I won't be going into depth
196on any of these tools since visualization is largely a subjective and
197creative experience, but remember to constantly explore new options as
198you repeat the process.
199
200Lastly, let's take a look at an example of data visualization. This
201example comes from a
202[[https://talent.works/2018/03/28/the-science-of-the-job-search-part-iii-61-of-entry-level-jobs-require-3-years-of-experience/][blog
203post written by Kushal Chakrabarti]] in 2018 about the percent of
204entry-level US jobs that require experience. *Figure 3* shows us an
205easy-to-digest picture of the data. We can quickly tell that only about
20612.5% of entry-level jobs don't require experience.
207
208This is the kind of result that easily describes the data for you.
209However, make sure to include an explanation of what the results mean.
210Don't let the reader assume what the data means, especially if it
211relates to a complex subject. /Tell a story/ about the data and why the
212results matter.
170Once an auditor has finally found the right data, KPIs, and tools, they must
171report these results so that actions can be taken. Performing in-depth data
172analysis is only useful if the results are understood by the audiences of the
173data. The best way to create this understanding is to visualize the results of
174the data. Let's take a look at some of the best options to visualize and report
175the results you've found.
176
177Some of the most popular commercial tools for visualization are Microsoft
178PowerBI and Tableau Desktop. However, other tools exist such as JMP, Plotly,
179Qlikview, Alteryx, or D3. Some require commercial licenses while others are
180simply free to use. For corporate data, you may want to make sure that the tool
181does not communicate any of the data outside the company (such as cloud
182storage). I won't be going into depth on any of these tools since visualization
183is largely a subjective and creative experience, but remember to constantly
184explore new options as you repeat the process.
185
186Lastly, let's take a look at an example of data visualization. This example
187comes from a [[https://talent.works/2018/03/28/the-science-of-the-job-search-part-iii-61-of-entry-level-jobs-require-3-years-of-experience/][blog post written by Kushal Chakrabarti]] in 2018 about the percent
188of entry-level US jobs that require experience. *Figure 3* shows us an
189easy-to-digest picture of the data. We can quickly tell that only about 12.5% of
190entry-level jobs don't require experience.
191
192This is the kind of result that easily describes the data for you. However, make
193sure to include an explanation of what the results mean. Don't let the reader
194assume what the data means, especially if it relates to a complex subject. /Tell
195a story/ about the data and why the results matter.
213196
214197* Wrap-Up
215198
216While this is not an all-encompassing program that you can just adopt
217into your department, it should be enough to get anyone started on the
218process of understanding and implementing data analytics. Always
219remember to continue learning and exploring new options as your
220processes grow and evolve.
199While this is not an all-encompassing program that you can just adopt into your
200department, it should be enough to get anyone started on the process of
201understanding and implementing data analytics. Always remember to continue
202learning and exploring new options as your processes grow and evolve.
content/blog/2019-12-03-the-ansoff-matrix.org +83 −95
@@ -5,118 +5,106 @@
55
66* Overview
77
8As the world of business evolves, managers must approach business
9planning and strategy with a contemporary mindset. According to Dess,
10McNamara, Eisner, and Lee, managers must be willing to adapt to the
11modern business environment by going beyond "'incremental management',
12whereby they view their job as making a series of small, minor changes
13to improve the efficiency of the firm's operations"(2019).
14
15One reason that strategic management is crucial is because most
16businesses that fail in the United States each year fail due to a lack
17of strategic focus or direction(2019). The rate of failure for
18businesses with poor strategies shows that strategic planning and
19management are crucial to a business's strength and longevity, injecting
20the critical factors of growth and direction into a company's business
21plan.
22
23One of the most significant strategic planning and management frameworks
24that companies can use is the
25[[https://en.wikipedia.org/wiki/Ansoff_matrix][Ansoff Matrix]]. While
26this framework has unique purposes and use-cases, it can effectively
27help an organization grow and compete. Specifically, the Ansoff matrix
28is one of the most effective frameworks for companies who want to focus
29on increasing sales revenue or profitability(2019).
30
31This framework uses a two-by-two figure to show the four strategic
32options for companies to use in this framework: market penetration,
33market development, product development, and diversification (see
34*Figure 1*). The x-axis of the matrix focuses on the firm's markets and
35also determines if the firm is looking to enter new markets or innovate
36in its current markets. The y-axis of the matrix focuses on the firm's
37products and determines if the firm wants to pursue strategies around
38their existing products or explore new products.
8As the world of business evolves, managers must approach business planning and
9strategy with a contemporary mindset. According to Dess, McNamara, Eisner, and
10Lee, managers must be willing to adapt to the modern business environment by
11going beyond "'incremental management', whereby they view their job as making a
12series of small, minor changes to improve the efficiency of the firm's
13operations"(2019).
14
15One reason that strategic management is crucial is because most businesses that
16fail in the United States each year fail due to a lack of strategic focus or
17direction(2019). The rate of failure for businesses with poor strategies shows
18that strategic planning and management are crucial to a business's strength and
19longevity, injecting the critical factors of growth and direction into a
20company's business plan.
21
22One of the most significant strategic planning and management frameworks that
23companies can use is the [[https://en.wikipedia.org/wiki/Ansoff_matrix][Ansoff Matrix]]. While this framework has unique purposes
24and use-cases, it can effectively help an organization grow and compete.
25Specifically, the Ansoff matrix is one of the most effective frameworks for
26companies who want to focus on increasing sales revenue or profitability(2019).
27
28This framework uses a two-by-two figure to show the four strategic options for
29companies to use in this framework: market penetration, market development,
30product development, and diversification (see *Figure 1*). The x-axis of the
31matrix focuses on the firm's markets and also determines if the firm is looking
32to enter new markets or innovate in its current markets. The y-axis of the
33matrix focuses on the firm's products and determines if the firm wants to pursue
34strategies around their existing products or explore new products.
3935
4036* Strategic Options
4137
4238** Market Penetration
4339
44The most straightforward strategy in the Ansoff matrix is to focus on
45existing products in existing markets, also known as market
46penetration(2019). Companies such as Coca-Cola have used market
47penetration successfully by investing a lot of money to get further
48value out of their current markets. Coca-Cola does this by introducing
49new features such as Christmas-themed bottles, personal names on the
50bottles, and other marketing schemes.
40The most straightforward strategy in the Ansoff matrix is to focus on existing
41products in existing markets, also known as market penetration(2019). Companies
42such as Coca-Cola have used market penetration successfully by investing a lot
43of money to get further value out of their current markets. Coca-Cola does this
44by introducing new features such as Christmas-themed bottles, personal names on
45the bottles, and other marketing schemes.
5146
5247** Market Development
5348
54Market development extends existing products into new markets in an
55attempt to increase the number of buyers. One interesting way that
56Coca-Cola used this strategy comes from the stigma that Diet Coke is a
57woman's drink(2019). Coca-Cola introduced Coca-Cola Zero, which
58contained the same nutritional content as Diet Coke, but was packaged in
59a dark black can to appear more "manly"(2019).
49Market development extends existing products into new markets in an attempt to
50increase the number of buyers. One interesting way that Coca-Cola used this
51strategy comes from the stigma that Diet Coke is a woman's drink(2019).
52Coca-Cola introduced Coca-Cola Zero, which contained the same nutritional
53content as Diet Coke, but was packaged in a dark black can to appear more
54"manly"(2019).
6055
6156** Product Development
6257
63Product development uses existing markets to introduce new products so
64that the firm can better meet customer needs(2019). The extreme end of
65diversification is home to companies such as Johnson & Johnson, a
66healthcare company that has developed a business portfolio of more than
6760,000 different products(2019). Johnson & Johnson's dedication to
68continuous diversification has led them to a balance sheet rating of
69"AAA", industry recognition for diversification, and increases in their
70investor dividends for 57 consecutive years(2019).
58Product development uses existing markets to introduce new products so that the
59firm can better meet customer needs(2019). The extreme end of diversification is
60home to companies such as Johnson & Johnson, a healthcare company that has
61developed a business portfolio of more than 60,000 different products(2019).
62Johnson & Johnson's dedication to continuous diversification has led them to a
63balance sheet rating of "AAA", industry recognition for diversification, and
64increases in their investor dividends for 57 consecutive years(2019).
7165
7266** Related Diversification
7367
74Diversification, the final strategy of the Ansoff Matrix, is more
75difficult than the others since it involves exploring both new markets
76and new products. Related diversification is a diversification strategy
77that closely relates to the firm's core business. Coca-Cola's best
78example of related diversification is its acquisition of Glaceau and
79Vitamin Water, which expanded their drinking lines of business(2019).
68Diversification, the final strategy of the Ansoff Matrix, is more difficult than
69the others since it involves exploring both new markets and new products.
70Related diversification is a diversification strategy that closely relates to
71the firm's core business. Coca-Cola's best example of related diversification is
72its acquisition of Glaceau and Vitamin Water, which expanded their drinking
73lines of business(2019).
8074
8175** Unrelated Diversification
8276
83Unrelated diversification is a diversification strategy that does not
84really relate to the firm's core business but still diversifies their
85business portfolio. A good example of this would be a coffee company who
86has decided to enter the market for bicycle sales. The main purpose of
87this strategy is to an extremely diverse company that will not go
88bankrupt if one market goes through difficult times. However, this
89requires a lot of independent skills and heavy investments since the
90company most likely cannot easily transfer knowledge between the markets
91they compete in.
77Unrelated diversification is a diversification strategy that does not really
78relate to the firm's core business but still diversifies their business
79portfolio. A good example of this would be a coffee company who has decided to
80enter the market for bicycle sales. The main purpose of this strategy is to an
81extremely diverse company that will not go bankrupt if one market goes through
82difficult times. However, this requires a lot of independent skills and heavy
83investments since the company most likely cannot easily transfer knowledge
84between the markets they compete in.
9285
9386* Requirements for Success
9487
95To use the Ansoff Matrix framework, managers need to formulate corporate
96goals and objectives. Without goals and direction, management frameworks
97do not present much practical utility. Further, the Ansoff Matrix
98requires the managers involved to make tactical decisions and create a
99path for the company to take toward their goals. Lastly, both the Ansoff
100Matrix needs to consider both internal and external perspectives
101throughout the strategy formulation process.
102
103One interesting probability is that companies will be using multiple
104strategic planning and management frameworks at the same time. While
105this may sound like it could crowd the management process, there are
106numerous reasons to do so. For example, the Ansoff Matrix and the
107Balanced Scorecard are relatively popular, and they cover entirely
108different parts of a company's strategy. Using the results from the
109Balanced Scorecard could inform a company of the potential product and
110market demands, such as from customer or supplier survey results, to
111help the company determine which Ansoff Matrix strategy to pursue.
112However, a combined approach at this level would require mature
113frameworks and focused managers who are able to strategize at a high
114level.
115
116Lastly, it should be noted that the author of the Ansoff matrix, Igor
117Ansoff, often used the term
118[[https://en.wikipedia.org/wiki/Analysis_paralysis][paralysis by
119analysis]] to explain the mistake of companies who overuse analysis and
120spend too much time planning. Companies need to understand the utility
121of a strategic management framework while ensuring that the company is
122poised to execute as efficiently as they have planned.
88To use the Ansoff Matrix framework, managers need to formulate corporate goals
89and objectives. Without goals and direction, management frameworks do not
90present much practical utility. Further, the Ansoff Matrix requires the managers
91involved to make tactical decisions and create a path for the company to take
92toward their goals. Lastly, both the Ansoff Matrix needs to consider both
93internal and external perspectives throughout the strategy formulation process.
94
95One interesting probability is that companies will be using multiple strategic
96planning and management frameworks at the same time. While this may sound like
97it could crowd the management process, there are numerous reasons to do so. For
98example, the Ansoff Matrix and the Balanced Scorecard are relatively popular,
99and they cover entirely different parts of a company's strategy. Using the
100results from the Balanced Scorecard could inform a company of the potential
101product and market demands, such as from customer or supplier survey results, to
102help the company determine which Ansoff Matrix strategy to pursue. However, a
103combined approach at this level would require mature frameworks and focused
104managers who are able to strategize at a high level.
105
106Lastly, it should be noted that the author of the Ansoff matrix, Igor Ansoff,
107often used the term [[https://en.wikipedia.org/wiki/Analysis_paralysis][paralysis by analysis]] to explain the mistake of companies
108who overuse analysis and spend too much time planning. Companies need to
109understand the utility of a strategic management framework while ensuring that
110the company is poised to execute as efficiently as they have planned.
content/blog/2019-12-16-password-security.org +74 −92
@@ -7,121 +7,103 @@
77
88** Why Does It Matter?
99
10Information security, including passwords and identities, has become one
11of the most important digital highlights of the last decade. With
12[[https://www.usatoday.com/story/money/2018/12/28/data-breaches-2018-billions-hit-growing-number-cyberattacks/2413411002/][billions
13of people affected by data breaches each year]], there's a greater need
14to introduce strong information security systems. If you think you've
15been part of a breach, or you want to check and see, you can use
16[[https://haveibeenpwned.com/][Have I Been Pwned]] to see if your email
17has been involved in any public breaches. Remember that there's a
18possibility that a company experienced a breach and did not report it to
19anyone.
10Information security, including passwords and identities, has become one of the
11most important digital highlights of the last decade. With [[https://www.usatoday.com/story/money/2018/12/28/data-breaches-2018-billions-hit-growing-number-cyberattacks/2413411002/][billions of people
12affected by data breaches each year]], there's a greater need to introduce strong
13information security systems. If you think you've been part of a breach, or you
14want to check and see, you can use [[https://haveibeenpwned.com/][Have I Been Pwned]] to see if your email has
15been involved in any public breaches. Remember that there's a possibility that a
16company experienced a breach and did not report it to anyone.
2017
2118** How Do I Protect Myself?
2219
23The first place to start with any personal security check-up is to
24gather a list of all the different websites, apps, or programs that
25require you to have login credentials. Optionally, once you know where
26your information is being stored, you can sort the list from the
27most-important items such as banks or government logins to less
28important items such as your favorite meme site. You will want to ensure
29that your critical logins are secure before getting to the others.
20The first place to start with any personal security check-up is to gather a list
21of all the different websites, apps, or programs that require you to have login
22credentials. Optionally, once you know where your information is being stored,
23you can sort the list from the most-important items such as banks or government
24logins to less important items such as your favorite meme site. You will want to
25ensure that your critical logins are secure before getting to the others.
3026
3127Once you think you have a good idea of all your different authentication
32methods, I recommend using a password manager such as
33[[https://bitwarden.com/][Bitwarden]]. Using a password manager allows
34you to automatically save your logins, create randomized passwords, and
35transfer passwords across devices. However, you'll need to memorize your
36"vault password" that allows you to open the password manager. It's
37important to make this something hard to guess since it would allow
38anyone who has it to access every password you've stored in there.
39
40Personally, I recommend using a
41[[https://en.wikipedia.org/wiki/Passphrase][passphrase]] instead of a
42[[https://en.wikipedia.org/wiki/Password][password]] for your vault
43password. Instead of using a string of characters (whether random or
44simple), use a phrase and add in symbols and a number. For example, your
45vault password could be =Racing-Alphabet-Gourd-Parrot3=. Swap the
46symbols out for whichever symbol you want, move the number around, and
47fine-tune the passphrase until you are confident that you can remember
48it whenever necessary.
49
50Once you've stored your passwords, make sure you continually check up on
51your account and make sure you aren't following bad password practices.
52Krebs on Security has a great
53[[https://krebsonsecurity.com/password-dos-and-donts/][blog post on
54password recommendations]]. Any time that a data breach happens, make
55sure you check to see if you were included, and if you need to reset any
56account passwords.
28methods, I recommend using a password manager such as [[https://bitwarden.com/][Bitwarden]]. Using a
29password manager allows you to automatically save your logins, create randomized
30passwords, and transfer passwords across devices. However, you'll need to
31memorize your "vault password" that allows you to open the password manager.
32It's important to make this something hard to guess since it would allow anyone
33who has it to access every password you've stored in there.
34
35Personally, I recommend using a [[https://en.wikipedia.org/wiki/Passphrase][passphrase]] instead of a [[https://en.wikipedia.org/wiki/Password][password]] for your vault
36password. Instead of using a string of characters (whether random or simple),
37use a phrase and add in symbols and a number. For example, your vault password
38could be =Racing-Alphabet-Gourd-Parrot3=. Swap the symbols out for whichever
39symbol you want, move the number around, and fine-tune the passphrase until you
40are confident that you can remember it whenever necessary.
41
42Once you've stored your passwords, make sure you continually check up on your
43account and make sure you aren't following bad password practices. Krebs on
44Security has a great [[https://krebsonsecurity.com/password-dos-and-donts/][blog post on password recommendations]]. Any time that a data
45breach happens, make sure you check to see if you were included, and if you need
46to reset any account passwords.
5747
5848* Developers
5949
6050** What Are the Basic Requirements?
6151
62When developing any password-protected application, there are a few
63basic rules that anyone should follow even if they do not follow any
64official guidelines such as NIST. The foremost practice is to require
65users to use passwords that are at least 8 characters and cannot easily
66be guessed. This sounds extremely simple, but it requires quite a few
67different strategies. First, the application should check the potential
68passwords against a dictionary of insecure passwords such =password=,
69=1234abc=, or =application_name=.
70
71Next, the application should offer guidance on the strength of passwords
72being entered during enrollment. Further, NIST officially recommends
73*not* implementing any composition rules that make passwords hard to
74remember (e.g. passwords with letters, numbers, and special characters)
75and instead encouraging the use of long pass phrases which can include
76spaces. It should be noted that to be able to keep spaces within
77passwords, all unicode characters should be supported, and passwords
78should not be truncated.
52When developing any password-protected application, there are a few basic rules
53that anyone should follow even if they do not follow any official guidelines
54such as NIST. The foremost practice is to require users to use passwords that
55are at least 8 characters and cannot easily be guessed. This sounds extremely
56simple, but it requires quite a few different strategies. First, the application
57should check the potential passwords against a dictionary of insecure passwords
58such =password=, =1234abc=, or =application_name=.
59
60Next, the application should offer guidance on the strength of passwords being
61entered during enrollment. Further, NIST officially recommends *not*
62implementing any composition rules that make passwords hard to remember (e.g.
63passwords with letters, numbers, and special characters) and instead encouraging
64the use of long pass phrases which can include spaces. It should be noted that
65to be able to keep spaces within passwords, all unicode characters should be
66supported, and passwords should not be truncated.
7967
8068** What Does NIST Recommend?
8169
82The National Institute of Standards and Technology
83([[https://www.nist.gov][NIST]]) in the US Department of Commerce
84regularly publishes information around information security and digital
85identity guidelines. Recently, NIST published
86[[https://pages.nist.gov/800-63-3/sp800-63b.html][Special Publication
70The National Institute of Standards and Technology ([[https://www.nist.gov][NIST]]) in the US Department
71of Commerce regularly publishes information around information security and
72digital identity guidelines. Recently, NIST published [[https://pages.nist.gov/800-63-3/sp800-63b.html][Special Publication
8773800-63b]]: Digital Identity Guidelines and Authentication and Lifecycle
8874Management.
8975
9076#+begin_quote
91A Memorized Secret authenticator - commonly referred to as a password
92or, if numeric, a PIN - is a secret value intended to be chosen and
93memorized by the user. Memorized secrets need to be of sufficient
94complexity and secrecy that it would be impractical for an attacker to
95guess or otherwise discover the correct secret value. A memorized secret
96is something you know.
77A Memorized Secret authenticator - commonly referred to as a password or, if
78numeric, a PIN - is a secret value intended to be chosen and memorized by the
79user. Memorized secrets need to be of sufficient complexity and secrecy that it
80would be impractical for an attacker to guess or otherwise discover the correct
81secret value. A memorized secret is something you know.
9782
9883- NIST Special Publication 800-63B
9984#+end_quote
10085
101NIST offers a lot of guidance on passwords, but I'm going to highlight
102just a few of the important factors:
86NIST offers a lot of guidance on passwords, but I'm going to highlight just a
87few of the important factors:
10388
104- Require passwords to be a minimum of 8 characters (6 characters if
105 randomly generated and be generated using an approved random bit
106 generator).
107- Compare potential passwords against a list that contains values known
108 to be commonly-used, expected, or compromised.
89- Require passwords to be a minimum of 8 characters (6 characters if randomly
90 generated and be generated using an approved random bit generator).
91- Compare potential passwords against a list that contains values known to be
92 commonly-used, expected, or compromised.
10993- Offer guidance on password strength, such as a strength meter.
11094- Implement a rate-limiting mechanism to limit the number of failed
11195 authentication attempts for each user account.
112- Do not require composition rules for passwords and do not require
113 passwords to be changed periodically (unless compromised).
114- Allow pasting of user identification and passwords to facilitate the
115 use of password managers.
96- Do not require composition rules for passwords and do not require passwords to
97 be changed periodically (unless compromised).
98- Allow pasting of user identification and passwords to facilitate the use of
99 password managers.
116100- Allow users to view the password as it is being entered.
117- Use secure forms of communication and storage, including salting and
118 hashing passwords using a one-way key derivation function.
119
120NIST offers further guidance on other devices that require specific
121security policies, querying for passwords, and more. All the information
122discussed so far comes from
123[[https://pages.nist.gov/800-63-3/sp800-63b.html][NIST SP800-63b]] but
124NIST offers a lot of information on digital identities, enrollment,
125identity proofing, authentication, lifecycle management, federation, and
126assertions in the total [[https://pages.nist.gov/800-63-3/][NIST
127SP800-63 Digital Identity Guidelines]].
101- Use secure forms of communication and storage, including salting and hashing
102 passwords using a one-way key derivation function.
103
104NIST offers further guidance on other devices that require specific security
105policies, querying for passwords, and more. All the information discussed so far
106comes from [[https://pages.nist.gov/800-63-3/sp800-63b.html][NIST SP800-63b]] but NIST offers a lot of information on digital
107identities, enrollment, identity proofing, authentication, lifecycle management,
108federation, and assertions in the total [[https://pages.nist.gov/800-63-3/][NIST SP800-63 Digital Identity
109Guidelines]].
content/blog/2020-01-25-linux-software.org +66 −87
@@ -7,29 +7,27 @@
77
88** Etcher
99
10[[https://www.balena.io/etcher/][Etcher]] is a quick and easy way to
11burn ISO images to CDs and USB devices. There are two different ways you
12can install this program. First, you can navigate to the
13[[https://www.balena.io/etcher/][official website]] and download the
14AppImage file, which can run without installation.
10[[https://www.balena.io/etcher/][Etcher]] is a quick and easy way to burn ISO images to CDs and USB devices. There
11are two different ways you can install this program. First, you can navigate to
12the [[https://www.balena.io/etcher/][official website]] and download the AppImage file, which can run without
13installation.
1514
16However, AppImage files are not executable by default, so you'll either
17need to right-click to open the properties of the file and click the
18"Allow executing file as program" box in the Permissions tab or use the
19following command:
15However, AppImage files are not executable by default, so you'll either need to
16right-click to open the properties of the file and click the "Allow executing
17file as program" box in the Permissions tab or use the following command:
2018
2119#+begin_src sh
2220chmod u+x FILE_NAME
2321#+end_src
2422
25If you don't like AppImage files or just prefer repositories, you can
26use the following commands to add the author's repository and install it
27through the command-line only.
23If you don't like AppImage files or just prefer repositories, you can use the
24following commands to add the author's repository and install it through the
25command-line only.
2826
2927First, you'll have to echo the repo and write it to a list file:
3028
3129#+begin_src sh
32echo "deb https://deb.etcher.io stable etcher" | sudo tee /etc/apt/sources.list.d/balena-etcher.list
30echo "deb https://deb.etcher.iostable etcher" | sudo tee /etc/apt/sources.list.d/balena-etcher.list
3331#+end_src
3432
3533Next, add the application keys to Ubuntu's keyring:
@@ -44,8 +42,7 @@ Finally, update the repositories and install the app.
4442sudo apt update && sudo apt install balena-etcher-electron
4543#+end_src
4644
47Using Arch, Manjaro, or another distro using the AUR? Use this command
48instead:
45Using Arch, Manjaro, or another distro using the AUR? Use this command instead:
4946
5047#+begin_src sh
5148sudo pacman -S etcher
@@ -53,11 +50,9 @@ sudo pacman -S etcher
5350
5451** Atom
5552
56[[https://atom.io][Atom]] is the self-proclaimed "hackable text editor
57for the 21st century". This text editor is made by GitHub,
58[[https://news.microsoft.com/2018/06/04/microsoft-to-acquire-github-for-7-5-billion/][now
59owned by Microsoft]], and has some of the best add-ons available to
60customize the layout and abilities of the app.
53[[https://atom.io][Atom]] is the self-proclaimed "hackable text editor for the 21st century". This
54text editor is made by GitHub, [[https://news.microsoft.com/2018/06/04/microsoft-to-acquire-github-for-7-5-billion/][now owned by Microsoft]], and has some of the best
55add-ons available to customize the layout and abilities of the app.
6156
6257First, add the Atom repository to your sources.
6358
@@ -71,26 +66,23 @@ Next, update your package listings and install atom.
7166sudo apt update && sudo apt install atom
7267#+end_src
7368
74If you have issues updating your packages with the Atom repository,
75you'll need to use the snap package described below instead of the
76repository. To remove the repository we just added, use this command:
69If you have issues updating your packages with the Atom repository, you'll need
70to use the snap package described below instead of the repository. To remove the
71repository we just added, use this command:
7772
7873#+begin_src sh
7974sudo add-apt-repository -r ppa:webupd8team/atom
8075#+end_src
8176
82You can also install Atom as a snap package, but it must be installed
83with the =--classic= flag. A
84[[https://language-bash.com/blog/how-to-snap-introducing-classic-confinement][full
85explanation is available]] if you'd like to read more about why you need
86the classic flag.
77You can also install Atom as a snap package, but it must be installed with the
78=--classic= flag. A [[https://language-bash.com/blog/how-to-snap-introducing-classic-confinement][full explanation is available]] if you'd like to read more
79about why you need the classic flag.
8780
8881#+begin_src sh
8982snap install atom --classic
9083#+end_src
9184
92Using Arch, Manjaro, or another distro using the AUR? Use this command
93instead:
85Using Arch, Manjaro, or another distro using the AUR? Use this command instead:
9486
9587#+begin_src sh
9688sudo pacman -S atom
@@ -98,28 +90,23 @@ sudo pacman -S atom
9890
9991** Visual Studio Code
10092
101[[https://code.visualstudio.com][Visual Studio Code]] is yet another
102fantastic choice for programming on Linux, especially if you need those
103extra add-ons to spice up your late-night coding sessions. The theme
104used in the screenshot is
105[[https://marketplace.visualstudio.com/items?itemName=EliverLara.mars][Mars]]
106by theme creator [[https://github.com/EliverLara][Eliver Lara]], who
107makes a ton of great themes for VS Code, Atom, and various Linux desktop
93[[https://code.visualstudio.com][Visual Studio Code]] is yet another fantastic choice for programming on Linux,
94especially if you need those extra add-ons to spice up your late-night coding
95sessions. The theme used in the screenshot is [[https://marketplace.visualstudio.com/items?itemName=EliverLara.mars][Mars]] by theme creator [[https://github.com/EliverLara][Eliver Lara]],
96who makes a ton of great themes for VS Code, Atom, and various Linux desktop
10897environments.
10998
110To install VS Code, you'll need to download the =.deb= file from the
111official website. Once you've downloaded the file, either double-click
112it to install through the Software Center or run the following command:
99To install VS Code, you'll need to download the =.deb= file from the official
100website. Once you've downloaded the file, either double-click it to install
101through the Software Center or run the following command:
113102
114103#+begin_src sh
115104sudo dpkg -i FILE_NAME.deb
116105#+end_src
117106
118You can also install VS Code as a snap package, but it must be installed
119with the =--classic= flag. A
120[[https://language-bash.com/blog/how-to-snap-introducing-classic-confinement][full
121explanation is available]] if you'd like to read more about why you need
122the classic flag.
107You can also install VS Code as a snap package, but it must be installed with
108the =--classic= flag. A [[https://language-bash.com/blog/how-to-snap-introducing-classic-confinement][full explanation is available]] if you'd like to read more
109about why you need the classic flag.
123110
124111#+begin_src sh
125112snap install code --classic
@@ -134,19 +121,18 @@ sudo pacman -S yay binutils make gcc pkg-config fakeroot yay -S visual-studio-co
134121
135122** GNOME Tweaks
136123
137[[https://gitlab.gnome.org/GNOME/gnome-tweaks][Gnome Tweaks]] is the
138ultimate tool to use if you want to customize your GNOME desktop
139environment. This is how you can switch application themes (GTK), shell
140themes, icons, fonts, and more. To install GNOME Tweaks on Ubuntu, you
141just need to install the official package.
124[[https://gitlab.gnome.org/GNOME/gnome-tweaks][Gnome Tweaks]] is the ultimate tool to use if you want to customize your GNOME
125desktop environment. This is how you can switch application themes (GTK), shell
126themes, icons, fonts, and more. To install GNOME Tweaks on Ubuntu, you just need
127to install the official package.
142128
143129#+begin_src sh
144130sudo apt install gnome-tweaks
145131#+end_src
146132
147If you've installed Manjaro or Arch with Gnome, you should have the
148tweak tool pre-installed. If you're on Fedora, this tool is available as
149an official package:
133If you've installed Manjaro or Arch with Gnome, you should have the tweak tool
134pre-installed. If you're on Fedora, this tool is available as an official
135package:
150136
151137#+begin_src sh
152138sudo dnf install gnome-tweaks
@@ -154,14 +140,12 @@ sudo dnf install gnome-tweaks
154140
155141** Steam
156142
157[[https://steampowered.com][Steam]] is one of the most popular gaming
158libraries for computers and is one of the main reasons that many people
159have been able to switch to Linux in recent years, thanks to Steam
160Proton, which makes it easier to play games not officially created for
161Linux platforms.
143[[https://steampowered.com][Steam]] is one of the most popular gaming libraries for computers and is one of
144the main reasons that many people have been able to switch to Linux in recent
145years, thanks to Steam Proton, which makes it easier to play games not
146officially created for Linux platforms.
162147
163To install Steam on Ubuntu, you just need to install the official
164package.
148To install Steam on Ubuntu, you just need to install the official package.
165149
166150#+begin_src sh
167151sudo apt install steam-installer
@@ -193,34 +177,32 @@ Finally, install the program:
193177sudo pacman -S steam
194178#+end_src
195179
196[[./2020-01-26-steam-on-ntfs-drives.html][Problem Launching Steam Games?
197Click Here.]]
180[[./2020-01-26-steam-on-ntfs-drives.html][Problem Launching Steam Games? Click Here.]]
198181
199182* Command-Line Packages
200183
201184** neofetch
202185
203[[https://github.com/dylanaraps/neofetch][Neofetch]] is a customizable
204tool used in the command-line to show system information. This is
205exceptionally useful if you want to see your system's information
206quickly without the clutter of some resource-heavy GUI apps.
186[[https://github.com/dylanaraps/neofetch][Neofetch]] is a customizable tool used in the command-line to show system
187information. This is exceptionally useful if you want to see your system's
188information quickly without the clutter of some resource-heavy GUI apps.
207189
208This is an official package if you're running Ubuntu 17.04 or later, so
209simply use the following command:
190This is an official package if you're running Ubuntu 17.04 or later, so simply
191use the following command:
210192
211193#+begin_src sh
212194sudo apt install neofetch
213195#+end_src
214196
215If you're running Ubuntu 16.10 or earlier, you'll have to use a series
216of commands:
197If you're running Ubuntu 16.10 or earlier, you'll have to use a series of
198commands:
217199
218200#+begin_src sh
219sudo add-apt-repository ppa:dawidd0811/neofetch; sudo apt update; sudo apt install neofetch
201sudo add-apt-repository ppa:dawidd0811/neofetch; sudo apt update; sudo apt
202install neofetch
220203#+end_src
221204
222Using Arch, Manjaro, or another distro using the AUR? Use this command
223instead:
205Using Arch, Manjaro, or another distro using the AUR? Use this command instead:
224206
225207#+begin_src sh
226208sudo pacman -S neofetch
@@ -228,31 +210,28 @@ sudo pacman -S neofetch
228210
229211** yt-dlp
230212
231[[https://github.com/yt-dlp/yt-dlp][yt-dlp]] is an extremely handy
232command-line tool that allows you to download video or audio files from
233various websites, such as YouTube. There are a ton of different options
234when running this package, so be sure to run =yt-dlp --help= first to
235look through everything you can do (or give up and search for the best
213[[https://github.com/yt-dlp/yt-dlp][yt-dlp]] is an extremely handy command-line tool that allows you to download video
214or audio files from various websites, such as YouTube. There are a ton of
215different options when running this package, so be sure to run =yt-dlp --help=
216first to look through everything you can do (or give up and search for the best
236217config online).
237218
238While this shouldn't be a problem for most users, yt-dlp requires Python
2392.6, 2.7, or 3.2+ to work correctly, so install Python if you don't have
240it already. You can check to see if you have Python installed by
241running:
219While this shouldn't be a problem for most users, yt-dlp requires Python 2.6,
2202.7, or 3.2+ to work correctly, so install Python if you don't have it already.
221You can check to see if you have Python installed by running:
242222
243223#+begin_src sh
244224python -V
245225#+end_src
246226
247To get the youtube-dl package, simply curl the URL and output the
248results.
227To get the youtube-dl package, simply curl the URL and output the results.
249228
250229#+begin_src sh
251sudo curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp
230sudo curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o
231/usr/local/bin/yt-dlp
252232#+end_src
253233
254Finally, make the file executable so that it can be run from the
255command-line.
234Finally, make the file executable so that it can be run from the command-line.
256235
257236#+begin_src sh
258237sudo chmod a+rx /usr/local/bin/yt-dlp
content/blog/2020-01-26-steam-on-ntfs.org +28 −30
@@ -5,28 +5,28 @@
55
66* Auto-Mount Steam Drives
77
8If you want to see how to install Steam on Linux, see my other post:
9[[../linux-software/][Linux Software]].
8If you want to see how to install Steam on Linux, see my other post: [[../linux-software/][Linux
9Software]].
1010
11Are you having trouble launching games, even though they've installed
12correctly? This may happen if you're storing your games on an
13NTFS-formatted drive. This shouldn't be an issue if you're storing your
14games on the same drive that Steam is on, but some gamers prefer to put
15Steam on their main drive and game files on another SSD or HDD.
11Are you having trouble launching games, even though they've installed correctly?
12This may happen if you're storing your games on an NTFS-formatted drive. This
13shouldn't be an issue if you're storing your games on the same drive that Steam
14is on, but some gamers prefer to put Steam on their main drive and game files on
15another SSD or HDD.
1616
17To fix this problem, you'll need to try a few things. First, you'll need
18to install the =ntfs-3g= package, which is meant for better
19interoperability with Linux.
17To fix this problem, you'll need to try a few things. First, you'll need to
18install the =ntfs-3g= package, which is meant for better interoperability with
19Linux.
2020
2121#+begin_src sh
2222sudo apt install ntfs-3g
2323#+end_src
2424
25Next, you should set up the =/etc/fstab= file to automatically mount
26your drives on boot. To automatically mount your drives when the
27computer boots up, you'll have to create the folders you want to mount
28your drive to first. I store mine in the =/mnt= folder using names that
29I'll recognize, but you can create your folders wherever you want.
25Next, you should set up the =/etc/fstab= file to automatically mount your drives
26on boot. To automatically mount your drives when the computer boots up, you'll
27have to create the folders you want to mount your drive to first. I store mine
28in the =/mnt= folder using names that I'll recognize, but you can create your
29folders wherever you want.
3030
3131#+begin_src sh
3232mkdir /path/to/folder
@@ -38,14 +38,13 @@ For example:
3838mkdir /mnt/steam_library
3939#+end_src
4040
41To automatically mount drives upon system boot, you will need to collect
42a few items. The UUID is the identification number connected to
43whichever drive you're using to store Steam games.
41To automatically mount drives upon system boot, you will need to collect a few
42items. The UUID is the identification number connected to whichever drive you're
43using to store Steam games.
4444
45Drives are usually labeled similar to =/dev/nvme0n1p1= or =/dev/sda1=,
46so you'll need to find the line in the output of the command below that
47correlates to your drive and copy the UUID over to the =/etc/fstab=
48file.
45Drives are usually labeled similar to =/dev/nvme0n1p1= or =/dev/sda1=, so you'll
46need to find the line in the output of the command below that correlates to your
47drive and copy the UUID over to the =/etc/fstab= file.
4948
5049#+begin_src sh
5150sudo blkid | grep UUID=
@@ -58,8 +57,8 @@ command:
5857id -u && id -g
5958#+end_src
6059
61Now that you have collected the necessary information, open the
62=/etc/fstab= file:
60Now that you have collected the necessary information, open the =/etc/fstab=
61file:
6362
6463#+begin_src sh
6564sudo nano /etc/fstab
@@ -72,9 +71,8 @@ Each drive you want to mount on boot should have its own line in the
7271UUID=B64E53824E5339F7 /mnt/steam_library ntfs-3g uid=1000,gid=1000 0 0
7372#+end_src
7473
75Now all you need to do is unmount your drive and re-mount it. You can
76unmount the drive by doing this (be sure to use the correct drive name
77here):
74Now all you need to do is unmount your drive and re-mount it. You can unmount
75the drive by doing this (be sure to use the correct drive name here):
7876
7977#+begin_src sh
8078sudo umount /dev/sdxX
@@ -86,6 +84,6 @@ You can re-mount all your drives by executing the following:
8684sudo mount -a
8785#+end_src
8886
89If you don't know what your drive name is, or you're nervous about
90unmounting and re-mounting, simply reboot your computer, and it will be
91done for you automatically.
87If you don't know what your drive name is, or you're nervous about unmounting
88and re-mounting, simply reboot your computer, and it will be done for you
89automatically.
content/blog/2020-02-09-cryptography-basics.org +127 −139
@@ -5,173 +5,161 @@
55
66* Similar Article Available
77
8If you haven't already, feel free to read my post on
9[[../aes-encryption/][AES Encryption]].
8If you haven't already, feel free to read my post on [[../aes-encryption/][AES Encryption]].
109
1110* What is Cryptography?
1211
1312In layman's terms, cryptography is a process that can change data from a
14readable format into an unreadable format (and vice-versa) through a
15series of processes and secrets. More technically, this is the Internet
16Security Glossary's definition:
13readable format into an unreadable format (and vice-versa) through a series of
14processes and secrets. More technically, this is the Internet Security
15Glossary's definition:
1716
1817#+begin_quote
19[Cryptography is] the mathematical science that deals with transforming
20data to render its meaning unintelligible (i.e., to hide its semantic
21content), prevent its undetected alteration, or prevent its unauthorized
22use. If the transformation is reversible, cryptography also deals with
23restoring encrypted data to an intelligible form.
24
25- [[https://tools.ietf.org/html/rfc2828][Internet Security Glossary
26 (2000)]]
18[Cryptography is] the mathematical science that deals with transforming data to
19render its meaning unintelligible (i.e., to hide its semantic content), prevent
20its undetected alteration, or prevent its unauthorized use. If the
21transformation is reversible, cryptography also deals with restoring encrypted
22data to an intelligible form.
23
24- [[https://tools.ietf.org/html/rfc2828][Internet Security Glossary (2000)]]
2725#+end_quote
2826
29Cryptography cannot offer protection against the loss of data; it simply
30offers encryption methods to protect data at-rest and data in-traffic.
31At a high-level, encrypted is when plaintext data is encrypted to
32ciphertext (a secure form of text that cannot be understood unless
33decrypted back to plaintext). The encryption process is completed
34through the use of a mathematical function that utilizes one or more
35values called keys to encrypt or decrypt the data.
27Cryptography cannot offer protection against the loss of data; it simply offers
28encryption methods to protect data at-rest and data in-traffic. At a high-level,
29encrypted is when plaintext data is encrypted to ciphertext (a secure form of
30text that cannot be understood unless decrypted back to plaintext). The
31encryption process is completed through the use of a mathematical function that
32utilizes one or more values called keys to encrypt or decrypt the data.
3633
3734* Key Elements of Cryptographic Systems
3835
39To create or evaluate a cryptographic system, you need to know the
40essential pieces to the system:
41
42- *Encryption Algorithm (Primitive):* A mathematical process that
43 encrypts and decrypts data.
44- *Encryption Key:* A string of bits used within the encryption
45 algorithm as the secret that allows successful encryption or
46 decryption of data.
47- *Key Length (Size):* The maximum number of bits within the encryption
48 key. It's important to remember that key size is regulated in many
49 countries.
50- *Message Digest:* A smaller, fixed-size bit string version of the
51 original message. This is practically infeasible to reverse, which is
52 why it's commonly used to verify integrity.
36To create or evaluate a cryptographic system, you need to know the essential
37pieces to the system:
38
39- *Encryption Algorithm (Primitive):* A mathematical process that encrypts and
40 decrypts data.
41- *Encryption Key:* A string of bits used within the encryption algorithm as the
42 secret that allows successful encryption or decryption of data.
43- *Key Length (Size):* The maximum number of bits within the encryption key.
44 It's important to remember that key size is regulated in many countries.
45- *Message Digest:* A smaller, fixed-size bit string version of the original
46 message. This is practically infeasible to reverse, which is why it's commonly
47 used to verify integrity.
5348
5449* Symmetric Systems (Secret Key Cryptography)
5550
56Symmetric cryptography utilizes a secret, bidirectional key to perform
57both encryption and decryption of the data. The most common
58implementation of symmetric cryptography is the Advanced Encryption
59Standard, which uses keys that are 128 bits to 256 bits in size. This
60standard came after the National Institute of Standards and Technology
61(NIST) decided to retire the Data Encryption Standard (DES) in 2001.
62
63Since brute force attacks strongly correlate with key length, the 56-bit
64key length of DES was considered insecure after it was publicly broken
65in under 24 hours. However, there is a modern implementation of DES
66called Triple DES where the DES method is applied three times to each
67data block.
68
69The main advantages to symmetric systems are the ease of use, since only
70one key is required for both encryption and decryption, and the
71simplicity of the algorithms. This helps with bulk data encryption that
72may unnecessarily waste time and power using asymmetric systems.
73
74However, symmetric systems have disadvantages to keep in mind. Since the
75key is private, it can be difficult to safely distribute keys to
76communication partners. Additionally, the key cannot be used to sign
77messages since it's necessary to keep the key private.
51Symmetric cryptography utilizes a secret, bidirectional key to perform both
52encryption and decryption of the data. The most common implementation of
53symmetric cryptography is the Advanced Encryption Standard, which uses keys that
54are 128 bits to 256 bits in size. This standard came after the National
55Institute of Standards and Technology (NIST) decided to retire the Data
56Encryption Standard (DES) in 2001.
57
58Since brute force attacks strongly correlate with key length, the 56-bit key
59length of DES was considered insecure after it was publicly broken in under 24
60hours. However, there is a modern implementation of DES called Triple DES where
61the DES method is applied three times to each data block.
62
63The main advantages to symmetric systems are the ease of use, since only one key
64is required for both encryption and decryption, and the simplicity of the
65algorithms. This helps with bulk data encryption that may unnecessarily waste
66time and power using asymmetric systems.
67
68However, symmetric systems have disadvantages to keep in mind. Since the key is
69private, it can be difficult to safely distribute keys to communication
70partners. Additionally, the key cannot be used to sign messages since it's
71necessary to keep the key private.
7872
7973* Asymmetric Systems (Public Key Cryptography)
8074
81Asymmetric cryptography utilizes two keys within the system: a secret
82key that is privately-held and a public key that can be distributed
83freely. The interesting aspect of asymmetric cryptography is that either
84key can be used to encrypt the data, there's no rule that dictates which
85key must be used for encryption. Once one key is used to encrypt the
86data, only the other key can be used to decrypt the data. This means
87that if the private key encrypts the data, only the public key can
88decrypt the data.
89
90An advantage of this system is that if you successfully decrypt data
91using one of the keys, you can be sure of the sender since only the
92other key could have encrypted the data.
93
94One of the major implementations of an asymmetric system is a digital
95signature. A digital signature can be generated using the sender's
96private key, or a one-way hash function and is used to provide assurance
97for the integrity and authenticity of the message. A couple common
98message digest algorithms are SHA-256 and SHA-512, which securely
99compress data and produce a 128-bit message digest.
100
101It should be noted that man-in-the-middle attacks are one of the risks
102with digital signatures and public keys. To combat this, applications
103often use a public key infrastructure (PKI) to independently
104authenticate the validity of signatures and keys.
105
106Due to the large key size and
107[[https://crypto.stackexchange.com/a/591][inefficient mathematical
108functions]] of asymmetric encryption, elliptical curve cryptography
109(ECC) is often used to increase security while using fewer resources.
75Asymmetric cryptography utilizes two keys within the system: a secret key that
76is privately-held and a public key that can be distributed freely. The
77interesting aspect of asymmetric cryptography is that either key can be used to
78encrypt the data, there's no rule that dictates which key must be used for
79encryption. Once one key is used to encrypt the data, only the other key can be
80used to decrypt the data. This means that if the private key encrypts the data,
81only the public key can decrypt the data.
82
83An advantage of this system is that if you successfully decrypt data using one
84of the keys, you can be sure of the sender since only the other key could have
85encrypted the data.
86
87One of the major implementations of an asymmetric system is a digital signature.
88A digital signature can be generated using the sender's private key, or a
89one-way hash function and is used to provide assurance for the integrity and
90authenticity of the message. A couple common message digest algorithms are
91SHA-256 and SHA-512, which securely compress data and produce a 128-bit message
92digest.
93
94It should be noted that man-in-the-middle attacks are one of the risks with
95digital signatures and public keys. To combat this, applications often use a
96public key infrastructure (PKI) to independently authenticate the validity of
97signatures and keys.
98
99Due to the large key size and [[https://crypto.stackexchange.com/a/591][inefficient mathematical functions]] of asymmetric
100encryption, elliptical curve cryptography (ECC) is often used to increase
101security while using fewer resources.
110102
111103* Applications of Cryptographic Systems
112104
113There are quite a few implementations of cryptographic systems around
114the world. Here are a few popular examples:
115
116*Transport Layer Security (TLS):* One of the most famous cryptographic
117solutions created is TLS, a session-layered or connection-layered
118internet protocol that allows for secure communications between browsers
119and servers. Using handshakes, peer negotiation, and authentication
120allows TLS to prevent eavesdropping and malicious transformation of
121data. The major reason for TLS popularity is that a major vulnerability
122was found in the SSL protocol in 2014. Instead of SSL, TLS can be used
123with HTTP to form HTTPS and is the preferred method for modern web
105There are quite a few implementations of cryptographic systems around the world.
106Here are a few popular examples:
107
108*Transport Layer Security (TLS):* One of the most famous cryptographic solutions
109created is TLS, a session-layered or connection-layered internet protocol that
110allows for secure communications between browsers and servers. Using handshakes,
111peer negotiation, and authentication allows TLS to prevent eavesdropping and
112malicious transformation of data. The major reason for TLS popularity is that a
113major vulnerability was found in the SSL protocol in 2014. Instead of SSL, TLS
114can be used with HTTP to form HTTPS and is the preferred method for modern web
124115development due to its increased security.
125116
126*Secure Hypertext Transfer Protocol (HTTPS):* An application layer
127protocol that allows for secure transport of data between servers and
128web clients. One of the unique parts of HTTPS is that it uses a secured
129port number instead of the default web port address.
117*Secure Hypertext Transfer Protocol (HTTPS):* An application layer protocol that
118allows for secure transport of data between servers and web clients. One of the
119unique parts of HTTPS is that it uses a secured port number instead of the
120default web port address.
130121
131*Virtual Private Network (VPN):* VPNs are made to securely extend a
132private network across public networks by utilizing an encrypted layered
133tunneling protocol paired with an authentication method, such as
134usernames and passwords. This technology originally allowed remote
135employees to access their company's data but have evolved into one of
136the top choices for anyone who wishes to mask their sensitive personal
137data.
122*Virtual Private Network (VPN):* VPNs are made to securely extend a private
123network across public networks by utilizing an encrypted layered tunneling
124protocol paired with an authentication method, such as usernames and passwords.
125This technology originally allowed remote employees to access their company's
126data but have evolved into one of the top choices for anyone who wishes to mask
127their sensitive personal data.
138128
139129*Internet Protocol Security (IPSec):* This protocol suite facilitates
140130communication between two or more hosts or subnets by authenticating and
141encrypting the data packets. IPSec is used in a lot of VPNs to establish
142the VPN connection through the transport and tunnel mode encryption
143methods. IPSec encrypts just the data portion of packets in the
144transport methods, but it encrypts both the data and headers in the
145tunnel method (introducing an additional header for authentication).
146
147*Secure Shell (SSH):* SSH is another network protocol used to protect
148network services by authenticating users through a secure channel. This
149protocol is often used for command-line (shell) functions such as remote
150shell commands, logins, and file transfers.
151
152*Kerberos:* Developed by MIT, Kerberos is a computer-network
153authentication protocol that works on the basis of tickets to allow
154nodes communicating over a non-secure network to prove their identity to
155one another securely. This is most commonly used in business
156environments when used as the authentication and encryption method for
157Windows Active Directory (AD).
131encrypting the data packets. IPSec is used in a lot of VPNs to establish the VPN
132connection through the transport and tunnel mode encryption methods. IPSec
133encrypts just the data portion of packets in the transport methods, but it
134encrypts both the data and headers in the tunnel method (introducing an
135additional header for authentication).
136
137*Secure Shell (SSH):* SSH is another network protocol used to protect network
138services by authenticating users through a secure channel. This protocol is
139often used for command-line (shell) functions such as remote shell commands,
140logins, and file transfers.
141
142*Kerberos:* Developed by MIT, Kerberos is a computer-network authentication
143protocol that works on the basis of tickets to allow nodes communicating over a
144non-secure network to prove their identity to one another securely. This is most
145commonly used in business environments when used as the authentication and
146encryption method for Windows Active Directory (AD).
158147
159148* Cybersecurity Controls
160149
161If you're someone who needs solutions on how to control risks associated
162with utilizing a crytograhpic system, start with a few basic controls:
150If you're someone who needs solutions on how to control risks associated with
151utilizing a crytograhpic system, start with a few basic controls:
163152
164- *Policies:* A policy on the use of cryptographic controls for
165 protection of information is implemented and is in accordance with
166 organizational objectives.
153- *Policies:* A policy on the use of cryptographic controls for protection of
154 information is implemented and is in accordance with organizational
155 objectives.
167156- *Key management:* A policy on the use, protection and lifetime of
168 cryptographic keys is implemented through the entire application
169 lifecycle.
170- *Key size:* The organization has researched the optimal key size for
171 their purposes, considering national laws, required processing power,
172 and longevity of the solution.
173- *Algorithm selection:* Implemented algorithms are sufficiently
174 appropriate for the business of the organization, robust, and align
175 with recommended guidelines.
176- *Protocol configuration:* Protocols have been reviewed and configured
177 suitable to the purpose of the business.
157 cryptographic keys is implemented through the entire application lifecycle.
158- *Key size:* The organization has researched the optimal key size for their
159 purposes, considering national laws, required processing power, and longevity
160 of the solution.
161- *Algorithm selection:* Implemented algorithms are sufficiently appropriate for
162 the business of the organization, robust, and align with recommended
163 guidelines.
164- *Protocol configuration:* Protocols have been reviewed and configured suitable
165 to the purpose of the business.
content/blog/2020-03-25-session-messenger.org +52 −60
@@ -5,46 +5,41 @@
55
66* Privacy Warning
77
8The company behind Session (Loki Foundation) is from Australia. If you
9didn't know, Australia has introduced
10[[https://parlinfo.aph.gov.au/parlInfo/download/legislation/bills/r6195_aspassed/toc_pdf/18204b01.pdf][legislation]]
11mandating companies comply with government requests to build backdoor
12access into applications. For more information, read my article on
13[[./2020-01-25-aes-encryption.html][AES Encryption]].
8The company behind Session (Loki Foundation) is from Australia. If you didn't
9know, Australia has introduced [[https://parlinfo.aph.gov.au/parlInfo/download/legislation/bills/r6195_aspassed/toc_pdf/18204b01.pdf][legislation]] mandating companies comply with
10government requests to build backdoor access into applications. For more
11information, read my article on [[./2020-01-25-aes-encryption.html][AES Encryption]].
1412
1513* About Session
1614
17[[https://getsession.org][Session]] is a private, cross-platform
18messaging app from the [[https://loki.foundation][Loki Foundation]]. As
19someone who has spent years looking for quality alternatives to major
20messaging apps, I was excited when I first heard about Session. Reading
21through [[https://arxiv.org/pdf/2002.04609.pdf][Session's white paper]],
22you can learn the technologies behind the Session app. Part of the
23security of Session comes from the Signal protocol, which was forked as
24the origin of Session.
15[[https://getsession.org][Session]] is a private, cross-platform messaging app from the [[https://loki.foundation][Loki Foundation]]. As
16someone who has spent years looking for quality alternatives to major messaging
17apps, I was excited when I first heard about Session. Reading through [[https://arxiv.org/pdf/2002.04609.pdf][Session's
18white paper]], you can learn the technologies behind the Session app. Part of the
19security of Session comes from the Signal protocol, which was forked as the
20origin of Session.
2521
2622#+begin_quote
27Session is an end-to-end encrypted messenger that removes sensitive
28metadata collection, and is designed for people who want privacy and
29freedom from any forms of surveillance.
23Session is an end-to-end encrypted messenger that removes sensitive metadata
24collection, and is designed for people who want privacy and freedom from any
25forms of surveillance.
3026#+end_quote
3127
3228In general, this app promises security through end-to-end encryption,
33decentralized onion routing, and private identities. The biggest change
34that the Loki Foundation has made to the Signal protocol is removing the
35need for a phone number. Instead, a random identification string is
36generated for any session you create. This means you can create a new
37session for each device if you want to, or link new devices with your
38ID.
29decentralized onion routing, and private identities. The biggest change that the
30Loki Foundation has made to the Signal protocol is removing the need for a phone
31number. Instead, a random identification string is generated for any session you
32create. This means you can create a new session for each device if you want to,
33or link new devices with your ID.
3934
40Since Session's website and white paper describe the details of
41Session's security, I'm going to focus on using the app in this post.
35Since Session's website and white paper describe the details of Session's
36security, I'm going to focus on using the app in this post.
4237
4338* Features
4439
45Since most people are looking for an alternative to a popular chat app,
46I am going to list out the features that Session has so that you are
47able to determine if the app would suit your needs:
40Since most people are looking for an alternative to a popular chat app, I am
41going to list out the features that Session has so that you are able to
42determine if the app would suit your needs:
4843
4944- Multiple device linking (via QR code or ID)
5045- App locking via device screen lock, password, or fingerprint
@@ -59,15 +54,15 @@ able to determine if the app would suit your needs:
5954
6055* Downloads
6156
62I have tested this app on Ubuntu 19.10, Android 10, macOS Monterey, and
63iOS 15. All apps have worked well without many issues.
57I have tested this app on Ubuntu 19.10, Android 10, macOS Monterey, and iOS 15.
58All apps have worked well without many issues.
6459
65Below is a brief overview of the Session app on Linux. To get this app,
66you'll need to go to the [[https://getsession.org/download/][Downloads]]
67page and click to link to the operating system you're using.
60Below is a brief overview of the Session app on Linux. To get this app, you'll
61need to go to the [[https://getsession.org/download/][Downloads]] page and click to link to the operating system
62you're using.
6863
69For Linux, it will download an AppImage that you'll need to enable with
70the following command:
64For Linux, it will download an AppImage that you'll need to enable with the
65following command:
7166
7267#+begin_src sh
7368sudo chmod u+x session-messenger-desktop-linux-x86_64-1.0.5.AppImage
@@ -75,42 +70,39 @@ sudo chmod u+x session-messenger-desktop-linux-x86_64-1.0.5.AppImage
7570
7671* Creating an Account
7772
78Once you've installed the app, simply run the app and create your unique
79Session ID. It will look something like this:
73Once you've installed the app, simply run the app and create your unique Session
74ID. It will look something like this:
8075=05af1835afdd63c947b47705867501d6373f486aa1ae05b1f2f3fcd24570eba608=.
8176
82You'll need to set a display name and, optionally, a password. If you
83set a password, you will need to enter it every time you open the app.
77You'll need to set a display name and, optionally, a password. If you set a
78password, you will need to enter it every time you open the app.
8479
8580* Start Messaging
8681
87Once you've created your account and set up your profile details, the
88next step is to start messaging other people. To do so, you'll need to
89share your Session ID with other people. From this point, it's fairly
90straightforward and acts like any other messaging app, so I won't dive
91into much detail here.
82Once you've created your account and set up your profile details, the next step
83is to start messaging other people. To do so, you'll need to share your Session
84ID with other people. From this point, it's fairly straightforward and acts like
85any other messaging app, so I won't dive into much detail here.
9286
9387** macOS
9488
95One key feature to note is that the desktop application now provides a
96helpful pop-up box explaining the process that Session uses to hide your
97IP address:
89One key feature to note is that the desktop application now provides a helpful
90pop-up box explaining the process that Session uses to hide your IP address:
9891
9992** iOS
10093
101The mobile app is quite simple and effective, giving you all the
102standard mobile messaging options you'd expect.
94The mobile app is quite simple and effective, giving you all the standard mobile
95messaging options you'd expect.
10396
10497* Potential Issues
10598
106I've discovered one annoying issue that would prevent from using this
107app regularly. On a mobile device, there have been issues with receiving
108messages on time. Even with battery optimization disabled and no network
109restrictions, Session notifications sometimes do not display until I
110open the app or the conversation itself and wait a few moments. This is
111actually one of the reasons I stopped using Signal (this seems fixed as
112of my updates in 2021/2022, so I wouldn't worry about this issue
113anymore).
114
115Looking for another messenger instead of Session? I recommend Signal,
116Matrix, and IRC.
99I've discovered one annoying issue that would prevent from using this app
100regularly. On a mobile device, there have been issues with receiving messages on
101time. Even with battery optimization disabled and no network restrictions,
102Session notifications sometimes do not display until I open the app or the
103conversation itself and wait a few moments. This is actually one of the reasons
104I stopped using Signal (this seems fixed as of my updates in 2021/2022, so I
105wouldn't worry about this issue anymore).
106
107Looking for another messenger instead of Session? I recommend Signal, Matrix,
108and IRC.
content/blog/2020-05-03-homelab.org +76 −88
@@ -5,27 +5,23 @@
55
66* What is a Homelab?
77
8Starting as a developer, I have largely stayed away from hardware-based
9hobbies (other than building a gaming desktop). However, as the
10quarantine for COVID-19 stretches out further and further, I found
11myself bored and in search of new hobbies. After spending the last few
12months browsing the [[https://www.reddit.com/r/homelab/][r/homelab]]
13subreddit, I decided it was time to jump in and try things out for
14myself.
8Starting as a developer, I have largely stayed away from hardware-based hobbies
9(other than building a gaming desktop). However, as the quarantine for COVID-19
10stretches out further and further, I found myself bored and in search of new
11hobbies. After spending the last few months browsing the [[https://www.reddit.com/r/homelab/][r/homelab]] subreddit, I
12decided it was time to jump in and try things out for myself.
1513
16Since I am a beginner and just recently graduated from college,
17everything I've done so far in my homelab is fairly low-budget.
14Since I am a beginner and just recently graduated from college, everything I've
15done so far in my homelab is fairly low-budget.
1816
1917* Hardware
2018
2119** Raspberry Pi 4
2220
23Luckily, I had actually purchased a
24[[https://www.raspberrypi.org/products/raspberry-pi-4-model-b/][Raspberry
25Pi 4]] before the quarantine started so that I could try to keep Plex
26Media Center running, even while my desktop computer was turned off. I
27started here, using the Pi to hold Plex and Pi-hole until I grew tired
28with the slow performance.
21Luckily, I had actually purchased a [[https://www.raspberrypi.org/products/raspberry-pi-4-model-b/][Raspberry Pi 4]] before the quarantine started
22so that I could try to keep Plex Media Center running, even while my desktop
23computer was turned off. I started here, using the Pi to hold Plex and Pi-hole
24until I grew tired with the slow performance.
2925
3026Here are the specifications for the Pi 4:
3127
@@ -37,12 +33,12 @@ Here are the specifications for the Pi 4:
3733
3834** Dell Optiplex 5040
3935
40Since I wasn't happy with the Pi as my main server, I turned to
41Craigslist. I know a lot of other homelabbers use Ebay, but I can't seem
42to ever trust it enough to purchase items on there. So I used Craigslist
43and found a Dell Optiplex 5040 desktop computer on sale for $90. While
44this computer might be underpowered, it was one of the few computers
45under $100 that was available during quarantine.
36Since I wasn't happy with the Pi as my main server, I turned to Craigslist. I
37know a lot of other homelabbers use Ebay, but I can't seem to ever trust it
38enough to purchase items on there. So I used Craigslist and found a Dell
39Optiplex 5040 desktop computer on sale for $90. While this computer might be
40underpowered, it was one of the few computers under $100 that was available
41during quarantine.
4642
4743Here are the specifications for the Dell Optiplex 5040:
4844
@@ -52,111 +48,103 @@ Here are the specifications for the Dell Optiplex 5040:
5248- Gigabit Ethernet
5349- 500GB Hard Drive
5450
55While this hardware would be awful for a work computer or a gaming rig,
56it turned out to be wonderful for my server purposes. The only
57limitation I have found so far is the CPU. The i3-6100 only has enough
58power for a single 4k video transcode at a time. I haven't tested more
59than three 1080p streams at a time, but the maximum amount of streams
60I've ever actually used is two.
51While this hardware would be awful for a work computer or a gaming rig, it
52turned out to be wonderful for my server purposes. The only limitation I have
53found so far is the CPU. The i3-6100 only has enough power for a single 4k video
54transcode at a time. I haven't tested more than three 1080p streams at a time,
55but the maximum amount of streams I've ever actually used is two.
6156
6257** WD easystore 10TB & 8TB
6358
64Application storage and temporary files are stored on the internal hard
65drive of the server, but all media files (movies, tv, games, books, etc)
66are stored externally on my WD easystore hard drive. Creating auto-boot
67configurations in the =/etc/fstab= file on my server allows the hard
68drives to automatically mount whenever I need to restart my server.
59Application storage and temporary files are stored on the internal hard drive of
60the server, but all media files (movies, tv, games, books, etc) are stored
61externally on my WD easystore hard drive. Creating auto-boot configurations in
62the =/etc/fstab= file on my server allows the hard drives to automatically mount
63whenever I need to restart my server.
6964
7065#+begin_quote
71Update: In March 2022, I shucked the hard drives out of their external
72cases, put some Kapton tape on the third power pin to prevent power
73shutdowns, and stuck them inside my server tower using internal SATA
74cables.
66Update: In March 2022, I shucked the hard drives out of their external cases,
67put some Kapton tape on the third power pin to prevent power shutdowns, and
68stuck them inside my server tower using internal SATA cables.
7569#+end_quote
7670
7771** Netgear Unmanaged Switch
7872
79To manage all the ethernet cords used by my homelab, my desktop, and my
80living room media center, I purchased an 8-port gigabit ethernet switch
81for $50 at my local computer store. This is probably much more than I
82should have spent on an unmanaged switch, but I am comfortable with the
83choice.
73To manage all the ethernet cords used by my homelab, my desktop, and my living
74room media center, I purchased an 8-port gigabit ethernet switch for $50 at my
75local computer store. This is probably much more than I should have spent on an
76unmanaged switch, but I am comfortable with the choice.
8477
8578** TP-Link Managed Switch
8679
87Since I use the unmanaged switch to group all living room devices
88together, I use the managed switch to configure VLANs and secure my
89network.
80Since I use the unmanaged switch to group all living room devices together, I
81use the managed switch to configure VLANs and secure my network.
9082
9183** Arris TM1602A Modem & Sagecom Fast 5280 Router
9284
93My default modem and router, provided by my ISP, are fairly standard.
94The Arris modem supports DOCSIS 3.0, which is something that I
95definitely wanted as a minimum. The Sagecom router is also standard, no
96fancy bells or whistles. However, it does support DHCP and DHCPv6, which
97is something you can use to route all household traffic through a
98pi-hole or firewall.
85My default modem and router, provided by my ISP, are fairly standard. The Arris
86modem supports DOCSIS 3.0, which is something that I definitely wanted as a
87minimum. The Sagecom router is also standard, no fancy bells or whistles.
88However, it does support DHCP and DHCPv6, which is something you can use to
89route all household traffic through a pi-hole or firewall.
9990
10091** TP-Link EAP
10192
102In order to gain better control over the network, I use my own wireless
103access point instead of the one included in the Sagecom router above.
104Now I can control and organize all of my ethernet connections through
105the VLANs on the managed switch and wireless connections through the
106VLANS on the EAP.
93In order to gain better control over the network, I use my own wireless access
94point instead of the one included in the Sagecom router above. Now I can control
95and organize all of my ethernet connections through the VLANs on the managed
96switch and wireless connections through the VLANS on the EAP.
10797
10898** Generic Printer
10999
110The last piece to my homelab is a standard wireless printer. Nothing
111special here.
100The last piece to my homelab is a standard wireless printer. Nothing special
101here.
112102
113103* Software
114104
115105** Ubuntu Server 20.04
116106
117While the 20.04 version of Ubuntu was just released, I always like to
118experiment with new features (and I don't mind breaking my system - it
119just gives me more experience learning how to fix things). So, I have
120Ubuntu Server 20.04 installed on the Dell Optiplex server and Ubuntu
121Server 19.10 installed on the Raspberry Pi. Once I find an acceptable
122use for the Pi, I will most likely switch the operating system.
107While the 20.04 version of Ubuntu was just released, I always like to experiment
108with new features (and I don't mind breaking my system - it just gives me more
109experience learning how to fix things). So, I have Ubuntu Server 20.04 installed
110on the Dell Optiplex server and Ubuntu Server 19.10 installed on the Raspberry
111Pi. Once I find an acceptable use for the Pi, I will most likely switch the
112operating system.
123113
124114** Docker
125115
126I am /very/ new to Docker, but I have had a lot of fun playing with it
127so far. Docker is used to create containers that can hold all the
128contents of a system without interfering with other software on the same
129system. So far, I have successfully installed pi-hole, GitLab, Gogs, and
130Nextcloud in containers. However, I opted to delete all of those so that
131I can reconfigure them more professionally at a later time.
116I am /very/ new to Docker, but I have had a lot of fun playing with it so far.
117Docker is used to create containers that can hold all the contents of a system
118without interfering with other software on the same system. So far, I have
119successfully installed pi-hole, GitLab, Gogs, and Nextcloud in containers.
120However, I opted to delete all of those so that I can reconfigure them more
121professionally at a later time.
132122
133123** Plex Media Server
134124
135Plex is a media center software that allows you to organize your movies,
136TV shows, music, photos, and videos automatically. It will even download
137metadata for you so that you can easily browse these collections.
125Plex is a media center software that allows you to organize your movies, TV
126shows, music, photos, and videos automatically. It will even download metadata
127for you so that you can easily browse these collections.
138128
139129** Pi-hole
140130
141Pi-hole is an alternative ad-blocker that runs at the DNS level,
142allowing you to block traffic when it hits your network, so that you can
143reject any traffic you deem to be bad. Pi-hole uses blacklists and
144whitelists to decide which traffic block and, luckily, there are a lot
145of pre-made lists out there on Reddit, GitHub, etc.
131Pi-hole is an alternative ad-blocker that runs at the DNS level, allowing you to
132block traffic when it hits your network, so that you can reject any traffic you
133deem to be bad. Pi-hole uses blacklists and whitelists to decide which traffic
134block and, luckily, there are a lot of pre-made lists out there on Reddit,
135GitHub, etc.
146136
147137** Nextcloud
148138
149While I had trouble with the Docker version of Nextcloud, I was very
150successful when setting up the snap version. Using this, I was able to
151map Nextcloud to a subdomain of a domain I own in Namecheap.
152Additionally, Nextcloud has an integration with Let's Encrypt that
153allows me to issue certificates automatically to any new domain I
154authorize.
139While I had trouble with the Docker version of Nextcloud, I was very successful
140when setting up the snap version. Using this, I was able to map Nextcloud to a
141subdomain of a domain I own in Namecheap. Additionally, Nextcloud has an
142integration with Let's Encrypt that allows me to issue certificates
143automatically to any new domain I authorize.
155144
156145** Webmin
157146
158To monitor my servers, and the processes running on them, I use the
159Webmin dashboard. This was fairly painless to set up, and I currently
160access it straight through the server's IP address. In the future, I
161will be looking to configure Webmin to use a custom domain just like
162Nextcloud.
147To monitor my servers, and the processes running on them, I use the Webmin
148dashboard. This was fairly painless to set up, and I currently access it
149straight through the server's IP address. In the future, I will be looking to
150configure Webmin to use a custom domain just like Nextcloud.
content/blog/2020-05-19-customizing-ubuntu.org +80 −96
@@ -10,145 +10,127 @@ For inspiration on designing your *nix computer, check out the
1010
1111* Customizing Ubuntu
1212
13New to Linux and want to add a personal touch to your machine? One of
14the best perks of Linux is that it is *extremely* customizable. You can
15change the styles of the windows, shell (status bars/docks), icons,
16fonts, terminals, and more.
13New to Linux and want to add a personal touch to your machine? One of the best
14perks of Linux is that it is *extremely* customizable. You can change the styles
15of the windows, shell (status bars/docks), icons, fonts, terminals, and more.
1716
18In this post, I'm going to go through customization on Ubuntu 20.04
19(GNOME) since most new users tend to choose Ubuntu-based distros. If
20you've found a way to install Arch with i3-gaps, I'm assuming you know
21how to find more advanced tutorials out there on customizations.
17In this post, I'm going to go through customization on Ubuntu 20.04 (GNOME)
18since most new users tend to choose Ubuntu-based distros. If you've found a way
19to install Arch with i3-gaps, I'm assuming you know how to find more advanced
20tutorials out there on customizations.
2221
2322** Required Tools
2423
25Ubuntu 20.04 ships with the default desktop environment
26[[https://www.gnome.org/][Gnome]], which includes the handy
27=gnome-tweaks= tool to quickly change designs. To install this, just
24Ubuntu 20.04 ships with the default desktop environment [[https://www.gnome.org/][Gnome]], which includes
25the handy =gnome-tweaks= tool to quickly change designs. To install this, just
2826open your terminal and enter the following command:
2927
3028#+begin_src sh
3129sudo apt install gnome-tweaks
3230#+end_src
3331
34After you've finished installing the tool, simply launch the Tweaks
35application, and you'll be able to access the various customization
36options available by default on Ubuntu. You might even like some of the
37pre-installed options.
32After you've finished installing the tool, simply launch the Tweaks application,
33and you'll be able to access the various customization options available by
34default on Ubuntu. You might even like some of the pre-installed options.
3835
3936** GNOME Application Themes
4037
41To change the themes applied to applications in GNOME, you will need to
42change the Applications dropdown in the Appearance section of Tweaks. To
43add more themes, you will have to find your preferred theme online and
44follow the steps below to have it show up in the Tweaks tool. While you
45may find themes anywhere, one of the most popular sites for GNOME themes
46is [[https://www.gnome-look.org/][gnome-look.org]]. This website
38To change the themes applied to applications in GNOME, you will need to change
39the Applications dropdown in the Appearance section of Tweaks. To add more
40themes, you will have to find your preferred theme online and follow the steps
41below to have it show up in the Tweaks tool. While you may find themes anywhere,
42one of the most popular sites for GNOME themes is [[https://www.gnome-look.org/][gnome-look.org]]. This website
4743contains themes for applications, shells, icons, and cursors.
4844
4945Steps to import themes into Tweaks:
5046
51471. Download the theme.
522. These files are usually compressed (.zip, .tar.gz, .tar.xz), so you
53 will need to extract the contents. This is easiest when opening the
54 file explorer, right-clicking the compressed file, and choosing
55 "Extract here."
482. These files are usually compressed (.zip, .tar.gz, .tar.xz), so you will need
49 to extract the contents. This is easiest when opening the file explorer,
50 right-clicking the compressed file, and choosing "Extract here."
56513. Move the theme folder to =/usr/share/themes/=. You can do so with the
5752 following command: =sudo mv theme-folder/ /usr/share/themes/=.
5853 - Icons and cursors will be moved to the =/usr/share/icons/= folder.
59 - Fonts will be moved to the =/usr/share/fonts/= folder
60 Alternatively, you can move them to the
61 =/usr/share/fonts/opentype/= or =/usr/share/fonts/opentype/=
62 folders, if you have a specific font type.
54 - Fonts will be moved to the =/usr/share/fonts/= folder Alternatively, you
55 can move them to the =/usr/share/fonts/opentype/= or
56 =/usr/share/fonts/opentype/= folders, if you have a specific font type.
63574. Close tweaks if it is open. Re-open Tweaks and your new theme will be
64 available in the Applications dropdown in the Appearance section of
65 Tweaks.
58 available in the Applications dropdown in the Appearance section of Tweaks.
6659
67If the theme is not showing up after you've moved it into the themes
68folder, you may have uncompressed the folder into a sub-folder. You can
69check this by entering the theme folder and listing the contents:
60If the theme is not showing up after you've moved it into the themes folder, you
61may have uncompressed the folder into a sub-folder. You can check this by
62entering the theme folder and listing the contents:
7063
7164#+begin_src sh
7265cd /usr/share/themes/Mojave-Dark && ls -la
7366#+end_src
7467
75This is an example of what the contents of your theme folder should look
76like. If you just see another folder there, you should move that folder
77up into the =/usr/share/themes/= folder.
68This is an example of what the contents of your theme folder should look like.
69If you just see another folder there, you should move that folder up into the
70=/usr/share/themes/= folder.
7871
7972#+begin_src sh
80cinnamon COPYING gnome-shell gtk-2.0 gtk-3.0 index.theme metacity-1 plank xfwm4
73cinnamon COPYING gnome-shell gtk-2.0 gtk-3.0 index.theme metacity-1 plank xfwm4
8174#+end_src
8275
8376** GNOME Shell Themes
8477
85To change the appearance of the title bar, default dock, app menu, and
86other parts of the GNOME shell, you'll need to install the
87[[https://extensions.gnome.org/extension/19/user-themes/][user themes]]
88extension on [[https://extensions.gnome.org/][Gnome Extensions]]. To be
89able to install extensions, you will first need to install the browser
90extension that the website instructs you to. See this screenshot for the
91blue box with a link to the extension.
78To change the appearance of the title bar, default dock, app menu, and other
79parts of the GNOME shell, you'll need to install the [[https://extensions.gnome.org/extension/19/user-themes/][user themes]] extension on
80[[https://extensions.gnome.org/][Gnome Extensions]]. To be able to install extensions, you will first need to
81install the browser extension that the website instructs you to. See this
82screenshot for the blue box with a link to the extension.
9283
93After the browser extension is installed, you will need to install the
94native host connector:
84After the browser extension is installed, you will need to install the native
85host connector:
9586
9687#+begin_src sh
9788sudo apt install chrome-gnome-shell
9889#+end_src
9990
100Finally, you can go the
101[[https://extensions.gnome.org/extension/19/user-themes/][user themes]]
102extension page and click the installation button. This will enable the
103Shell option in Tweaks. Now you can move shell themes to the
104=/usr/share/themes= directory, using the same steps mentioned in the
105previous section, and enable the new theme in Tweaks.
91Finally, you can go the [[https://extensions.gnome.org/extension/19/user-themes/][user themes]] extension page and click the installation
92button. This will enable the Shell option in Tweaks. Now you can move shell
93themes to the =/usr/share/themes= directory, using the same steps mentioned in
94the previous section, and enable the new theme in Tweaks.
10695
10796** Icons & Cursors
10897
109Icons and cursors are installed exactly the same way, so I'm grouping
110these together in this post. Both of these items will need to follow the
111same process as installing themes, except you will want to move your
112font folders to the =/usr/share/icons/= directory instead.
98Icons and cursors are installed exactly the same way, so I'm grouping these
99together in this post. Both of these items will need to follow the same process
100as installing themes, except you will want to move your font folders to the
101=/usr/share/icons/= directory instead.
113102
114103** Fonts
115104
116Fonts are one of the overlooked parts of customization, but a good font
117can make the whole screen look different. For example, I have installed
118the [[https://github.com/IBM/plex/releases][IBM Plex]] fonts on my
119system. This follows the same process as installing themes, except you
120will want to move your font folders to the =/usr/share/fonts/= directory
105Fonts are one of the overlooked parts of customization, but a good font can make
106the whole screen look different. For example, I have installed the [[https://github.com/IBM/plex/releases][IBM Plex]]
107fonts on my system. This follows the same process as installing themes, except
108you will want to move your font folders to the =/usr/share/fonts/= directory
121109instead.
122110
123111** Terminal
124112
125If you spend a lot of time typing commands, you know how important the
126style and functionality of the terminal is. After spending a lot of time
127using the default GNOME terminal with
128[[https://en.wikipedia.org/wiki/Bash_(Unix_shell)][unix shell]], I
129decided to try some different options. I ended up choosing
130[[https://terminator-gtk3.readthedocs.io/en/latest/][Terminator]] with
131[[https://en.wikipedia.org/wiki/Z_shell][zsh]].
132
133Terminator is great if you need to open multiple terminals at one time
134by simply right-clicking and splitting the screen into as many terminals
135as you want. While this project hasn't been updated in a while,
136[[https://github.com/gnome-terminator/terminator/issues/1][it is coming
137under new development]]. However, this terminal is great and I haven't
138experienced any errors yet.
139
140For the shell choice, I decided to choose zsh after trying it out on a
141fresh Manjaro installation. Zsh is great if you like to change the
142themes of your terminal, include icons, or add plugins.
143
144The desktop uses the
145[[https://github.com/zsh-users/zsh-autosuggestions][zsh-autosuggestions]]
146to suggest past commands as you type. In addition, it suggests
147corrections if you misspell a command. Lastly, it uses the =af-magic=
148theme, which adds dashed lines between commands, moving the user@host
149tag to the right side of the terminal, and changes the colors. There are
150plenty of plugins and themes to choose from. Just figure out what you
151like and add it to your =~/.zshrc= file!
113If you spend a lot of time typing commands, you know how important the style and
114functionality of the terminal is. After spending a lot of time using the default
115GNOME terminal with [[https://en.wikipedia.org/wiki/Bash_(Unix_shell)][unix shell]], I decided to try some different options. I ended
116up choosing [[https://terminator-gtk3.readthedocs.io/en/latest/][Terminator]] with [[https://en.wikipedia.org/wiki/Z_shell][zsh]].
117
118Terminator is great if you need to open multiple terminals at one time by simply
119right-clicking and splitting the screen into as many terminals as you want.
120While this project hasn't been updated in a while, [[https://github.com/gnome-terminator/terminator/issues/1][it is coming under new
121development]]. However, this terminal is great and I haven't experienced any
122errors yet.
123
124For the shell choice, I decided to choose zsh after trying it out on a fresh
125Manjaro installation. Zsh is great if you like to change the themes of your
126terminal, include icons, or add plugins.
127
128The desktop uses the [[https://github.com/zsh-users/zsh-autosuggestions][zsh-autosuggestions]] to suggest past commands as you type.
129In addition, it suggests corrections if you misspell a command. Lastly, it uses
130the =af-magic= theme, which adds dashed lines between commands, moving the
131user@host tag to the right side of the terminal, and changes the colors. There
132are plenty of plugins and themes to choose from. Just figure out what you like
133and add it to your =~/.zshrc= file!
152134
153135*** Steps to Replicate My Terminal
154136
@@ -173,18 +155,20 @@ sudo apt install terminator
173155To install Oh My Zsh on Ubuntu:
174156
175157#+begin_src sh
176sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
158sh -c "$(curl -fsSL
159https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
177160#+end_src
178161
179162To install zsh-autosuggestions via Oh My Zsh:
180163
181164#+begin_src sh
182git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions
165git clone https://github.com/zsh-users/zsh-autosuggestions
166${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions
183167#+end_src
184168
185Then, add the following plugin wording to your =~/.zshrc= file (the
186default config usually has the =git= plugin activated, so just add any
187other plugins to the parentheses separated by a space):
169Then, add the following plugin wording to your =~/.zshrc= file (the default
170config usually has the =git= plugin activated, so just add any other plugins to
171the parentheses separated by a space):
188172
189173#+begin_src sh
190174nano ~/.zshrc
@@ -194,5 +178,5 @@ nano ~/.zshrc
194178plugins=(git zsh-autosuggestions)
195179#+end_src
196180
197Finally, you need to log out of your computer and log back in so your
198user shell can refresh.
181Finally, you need to log out of your computer and log back in so your user shell
182can refresh.
content/blog/2020-07-20-video-game-sales.org +18 −21
@@ -5,10 +5,8 @@
55
66* Background Information
77
8This dataset (obtained from
9[[https://www.kaggle.com/gregorut/videogamesales/data][Kaggle]])
10contains a list of video games with sales greater than 100,000 copies.
11It was generated by a scrape of vgchartz.com.
8This dataset (obtained from [[https://www.kaggle.com/gregorut/videogamesales/data][Kaggle]]) contains a list of video games with sales
9greater than 100,000 copies. It was generated by a scrape of vgchartz.com.
1210
1311Fields include:
1412
@@ -24,8 +22,7 @@ Fields include:
2422- Other_{Sales}: Sales in the rest of the world (in millions)
2523- Global_{Sales}: Total worldwide sales.
2624
27There are 16,598 records. 2 records were dropped due to incomplete
28information.
25There are 16,598 records. 2 records were dropped due to incomplete information.
2926
3027* Import the Data
3128
@@ -138,18 +135,18 @@ df3.sort_values(by=['Global_Sales'], ascending=False).head(5)
138135
139136* Discussion
140137
141The purpose of exploring datasets is to ask questions, answer questions,
142and discover intelligence that can be used to inform decision-making.
143So, what have we found in this dataset?
144
145Today we simply explored a publicly-available dataset to see what kind
146of information it contained. During that exploration, we found that
147video game sales peaked in 2006. That peak was largely due to Nintendo,
148who sold the top 5 games in 2006 and has a number of games in the top-10
149list for the years 1980-2020. Additionally, the top four platforms by
150global sales (Wii, NES, GB, DS) are owned by Nintendo.
151
152We didn't explore everything this dataset has to offer, but we can tell
153from a brief analysis that Nintendo seems to rule sales in the video
154gaming world. Further analysis could provide insight into which genres,
155regions, publishers, or world events are correlated with sales.
138The purpose of exploring datasets is to ask questions, answer questions, and
139discover intelligence that can be used to inform decision-making. So, what have
140we found in this dataset?
141
142Today we simply explored a publicly-available dataset to see what kind of
143information it contained. During that exploration, we found that video game
144sales peaked in 2006. That peak was largely due to Nintendo, who sold the top 5
145games in 2006 and has a number of games in the top-10 list for the years
1461980-2020. Additionally, the top four platforms by global sales (Wii, NES, GB,
147DS) are owned by Nintendo.
148
149We didn't explore everything this dataset has to offer, but we can tell from a
150brief analysis that Nintendo seems to rule sales in the video gaming world.
151Further analysis could provide insight into which genres, regions, publishers,
152or world events are correlated with sales.
content/blog/2020-07-26-business-analysis.org +57 −62
@@ -5,9 +5,9 @@
55
66* Background Information
77
8This project aims to help investors learn more about a random city in
9order to determine optimal locations for business investments. The data
10used in this project was obtained using Foursquare's developer API.
8This project aims to help investors learn more about a random city in order to
9determine optimal locations for business investments. The data used in this
10project was obtained using Foursquare's developer API.
1111
1212Fields include:
1313
@@ -16,13 +16,13 @@ Fields include:
1616- Venue Latitude
1717- Venue Longitude
1818
19There are 232 records found using the center of Lincoln as the area of
20interest with a radius of 10,000.
19There are 232 records found using the center of Lincoln as the area of interest
20with a radius of 10,000.
2121
2222* Import the Data
2323
24The first step is the simplest: import the applicable libraries. We will
25be using the libraries below for this project.
24The first step is the simplest: import the applicable libraries. We will be
25using the libraries below for this project.
2626
2727#+begin_src python
2828# Import the Python libraries we will be using
@@ -35,10 +35,10 @@ from pandas.io.json import json_normalize
3535from sklearn.cluster import KMeans
3636#+end_src
3737
38To begin our analysis, we need to import the data for this project. The
39data we are using in this project comes directly from the Foursquare
40API. The first step is to get the latitude and longitude of the city
41being studied (Lincoln, NE) and setting up the folium map.
38To begin our analysis, we need to import the data for this project. The data we
39are using in this project comes directly from the Foursquare API. The first step
40is to get the latitude and longitude of the city being studied (Lincoln, NE) and
41setting up the folium map.
4242
4343#+begin_src python
4444# Define the latitude and longitude, then map the results
@@ -49,11 +49,11 @@ map_LNK = folium.Map(location=[latitude, longitude], zoom_start=12)
4949map_LNK
5050#+end_src
5151
52Now that we have defined our city and created the map, we need to go get
53the business data. The Foursquare API will limit the results to 100 per
54API call, so we use our first API call below to determine the total
55results that Foursquare has found. Since the total results are 232, we
56perform the API fetching process three times (100 + 100 + 32 = 232).
52Now that we have defined our city and created the map, we need to go get the
53business data. The Foursquare API will limit the results to 100 per API call, so
54we use our first API call below to determine the total results that Foursquare
55has found. Since the total results are 232, we perform the API fetching process
56three times (100 + 100 + 32 = 232).
5757
5858#+begin_src python
5959# Foursquare API credentials
@@ -117,13 +117,12 @@ results3 = requests.get(url3).json()
117117
118118* Clean the Data
119119
120Now that we have our data in three separate dataframes, we need to
121combine them into a single dataframe and make sure to reset the index so
122that we have a unique ID for each business. The =get~categorytype~=
123function below will pull the categories and name from each business's
124entry in the Foursquare data automatically. Once all the data has been
125labeled and combined, the results are stored in the =nearby_venues=
126dataframe.
120Now that we have our data in three separate dataframes, we need to combine them
121into a single dataframe and make sure to reset the index so that we have a
122unique ID for each business. The =get~categorytype~= function below will pull
123the categories and name from each business's entry in the Foursquare data
124automatically. Once all the data has been labeled and combined, the results are
125stored in the =nearby_venues= dataframe.
127126
128127#+begin_src python
129128# This function will extract the category of the venue from the API dictionary
@@ -192,9 +191,9 @@ nearby_venues
192191
193192* Visualize the Data
194193
195We now have a complete, clean data set. The next step is to visualize
196this data onto the map we created earlier. We will be using folium's
197=CircleMarker()= function to do this.
194We now have a complete, clean data set. The next step is to visualize this data
195onto the map we created earlier. We will be using folium's =CircleMarker()=
196function to do this.
198197
199198#+begin_src python
200199# add markers to map
@@ -216,15 +215,14 @@ map_LNK
216215
217216* Clustering: /k-means/
218217
219To cluster the data, we will be using the /k-means/ algorithm. This
220algorithm is iterative and will automatically make sure that data points
221in each cluster are as close as possible to each other, while being as
222far as possible away from other clusters.
218To cluster the data, we will be using the /k-means/ algorithm. This algorithm is
219iterative and will automatically make sure that data points in each cluster are
220as close as possible to each other, while being as far as possible away from
221other clusters.
223222
224However, we first have to figure out how many clusters to use (defined
225as the variable /'k'/). To do so, we will use the next two functions to
226calculate the sum of squares within clusters and then return the optimal
227number of clusters.
223However, we first have to figure out how many clusters to use (defined as the
224variable /'k'/). To do so, we will use the next two functions to calculate the
225sum of squares within clusters and then return the optimal number of clusters.
228226
229227#+begin_src python
230228# This function will return the sum of squares found in the data
@@ -262,9 +260,9 @@ def optimal_number_of_clusters(wcss):
262260n = optimal_number_of_clusters(sum_of_squares)
263261#+end_src
264262
265Now that we have found that our optimal number of clusters is six, we
266need to perform k-means clustering. When this clustering occurs, each
267business is assigned a cluster number from 0 to 5 in the dataframe.
263Now that we have found that our optimal number of clusters is six, we need to
264perform k-means clustering. When this clustering occurs, each business is
265assigned a cluster number from 0 to 5 in the dataframe.
268266
269267#+begin_src python
270268# set number of clusters equal to the optimal number
@@ -304,12 +302,11 @@ map_clusters
304302
305303* Investigate Clusters
306304
307Now that we have figured out our clusters, let's do a little more
308analysis to provide more insight into the clusters. With the information
309below, we can see which clusters are more popular for businesses and
310which are less popular. The results below show us that clusters 0
311through 3 are popular, while clusters 4 and 5 are not very popular at
312all.
305Now that we have figured out our clusters, let's do a little more analysis to
306provide more insight into the clusters. With the information below, we can see
307which clusters are more popular for businesses and which are less popular. The
308results below show us that clusters 0 through 3 are popular, while clusters 4
309and 5 are not very popular at all.
313310
314311#+begin_src python
315312# Show how many venues are in each cluster
@@ -320,9 +317,9 @@ for x in range(0,6):
320317 print("---")
321318#+end_src
322319
323Our last piece of analysis is to summarize the categories of businesses
324within each cluster. With these results, we can clearly see that
325restaurants, coffee shops, and grocery stores are the most popular.
320Our last piece of analysis is to summarize the categories of businesses within
321each cluster. With these results, we can clearly see that restaurants, coffee
322shops, and grocery stores are the most popular.
326323
327324#+begin_src python
328325# Calculate how many venues there are in each category
@@ -348,19 +345,17 @@ with pd.option_context('display.max_rows', None, 'display.max_columns', None):
348345
349346* Discussion
350347
351In this project, we gathered location data for Lincoln, Nebraska, USA
352and clustered the data using the k-means algorithm in order to identify
353the unique clusters of businesses in Lincoln. Through these actions, we
354found that there are six unique business clusters in Lincoln and that
355two of the clusters are likely unsuitable for investors. The remaining
356four clusters have a variety of businesses, but are largely dominated by
357restaurants and grocery stores.
358
359Using this project, investors can now make more informed decisions when
360deciding the location and category of business in which to invest.
361
362Further studies may involve other attributes for business locations,
363such as population density, average wealth across the city, or crime
364rates. In addition, further studies may include additional location data
365and businesses by utilizing multiple sources, such as Google Maps and
366OpenStreetMap.
348In this project, we gathered location data for Lincoln, Nebraska, USA and
349clustered the data using the k-means algorithm in order to identify the unique
350clusters of businesses in Lincoln. Through these actions, we found that there
351are six unique business clusters in Lincoln and that two of the clusters are
352likely unsuitable for investors. The remaining four clusters have a variety of
353businesses, but are largely dominated by restaurants and grocery stores.
354
355Using this project, investors can now make more informed decisions when deciding
356the location and category of business in which to invest.
357
358Further studies may involve other attributes for business locations, such as
359population density, average wealth across the city, or crime rates. In addition,
360further studies may include additional location data and businesses by utilizing
361multiple sources, such as Google Maps and OpenStreetMap.
content/blog/2020-08-22-redirect-github-pages.org +44 −52
@@ -7,18 +7,17 @@
77
88** Step 1
99
10Add a new file CNAME to your GitHub Pages repository containing only one
11line: your top-level domain name. E.g.: =example.com=
10Add a new file CNAME to your GitHub Pages repository containing only one line:
11your top-level domain name. E.g.: =example.com=
1212
1313** Step 2
1414
1515[Optional] but highly recommended
1616
172.1: Remove all other top-level records (prefixed with @) of type A from
18your DNS configuration.
172.1: Remove all other top-level records (prefixed with @) of type A from your
18DNS configuration.
1919
202.2: Remove a CNAME record for the second-level domain www if it is
21present.
202.2: Remove a CNAME record for the second-level domain www if it is present.
2221
2322** Step 3
2423
@@ -41,48 +40,45 @@ immediately. They can take up to a full day to propagate.
4140
4241* Long answer
4342
44This issue has two sides. One is the DNS configuration itself. Another
45one is the way GitHub Pages will forward HTTP requests.
43This issue has two sides. One is the DNS configuration itself. Another one is
44the way GitHub Pages will forward HTTP requests.
4645
47We need to know a few things to understand what GitHub is trying to say
48in their documentation.
46We need to know a few things to understand what GitHub is trying to say in their
47documentation.
4948
5049** DNS Entry Types
5150
5251There are two types of DNS records which interest us: CNAME and A.
5352
54=A= is also known as =Apex= or sometimes as =root entry=. It forwards
55requests to a specified fixed IP address. =CNAME= entry forwards
56requests to a specified URL (actual valid plain text URL, not an IP
57address).
53=A= is also known as =Apex= or sometimes as =root entry=. It forwards requests
54to a specified fixed IP address. =CNAME= entry forwards requests to a specified
55URL (actual valid plain text URL, not an IP address).
5856
5957** DNS Load balancing
6058
61GitHub has one central URL address which accepts all DNS requests for
62GitHub Pages: =http://username.github.io=. That URL is resolved to
63different IP addresses based on your geographical location. Website
64hosted on GitHub Pages is a simple collection of =HTML=, =CSS= and =JS=
65files. GitHub distributes these files to different servers across the
66globe. So that when your browser sends a request from Europe, it
67receives data from a server in Europe. The same is valid for the
68requests from Asia and the USA.
59GitHub has one central URL address which accepts all DNS requests for GitHub
60Pages: =http://username.github.io=. That URL is resolved to different IP
61addresses based on your geographical location. Website hosted on GitHub Pages is
62a simple collection of =HTML=, =CSS= and =JS= files. GitHub distributes these
63files to different servers across the globe. So that when your browser sends a
64request from Europe, it receives data from a server in Europe. The same is valid
65for the requests from Asia and the USA.
6966
7067** What GitHub is trying to say
7168
72Since =A= records in DNS must contain IP addresses, and they must be
73either =185.199.108.153= or =185.199.109.153= or =185.199.110.153= or
74=185.199.111.153=, there is no way to forward requests to a server
75located somewhere in Europe or Asia. Your website hosted at GitHub Pages
76will be downloaded from a central GitHub Pages server. There is a minor
77risk that if GitHub Pages DNS servers (=x.x.x.153=) are down for some
78reason, all custom domains which use fixed GitHub Pages IP addresses
79will not be accessible (their DNS requests will not be resolvable).
80
81That is why GitHub strongly suggests to either use a second-level domain
82for your GitHub Pages (e.g. =blog.example.com=) or use a DNS service
83provider that supports a record type =ALIAS= that acts as =A= record but
84forwards request to a URL address (e.g. =username.github.io=) instead of
85a fixed IP address.
69Since =A= records in DNS must contain IP addresses, and they must be either
70=185.199.108.153= or =185.199.109.153= or =185.199.110.153= or
71=185.199.111.153=, there is no way to forward requests to a server located
72somewhere in Europe or Asia. Your website hosted at GitHub Pages will be
73downloaded from a central GitHub Pages server. There is a minor risk that if
74GitHub Pages DNS servers (=x.x.x.153=) are down for some reason, all custom
75domains which use fixed GitHub Pages IP addresses will not be accessible (their
76DNS requests will not be resolvable).
77
78That is why GitHub strongly suggests to either use a second-level domain for
79your GitHub Pages (e.g. =blog.example.com=) or use a DNS service provider that
80supports a record type =ALIAS= that acts as =A= record but forwards request to a
81URL address (e.g. =username.github.io=) instead of a fixed IP address.
8682
8783** How GitHub Pages treats HTTP requests
8884
@@ -101,12 +97,12 @@ curl --header "Host: example.com" http://185.199.108.153/
10197This way GitHub Pages servers know which user website to serve.
10298
10399#+begin_quote
104GitHub Pages server will automatically redirect HTTP requests to the
105top-level domain if your =CNAME= file contains =example.com= but
106=www.example.com= is requested.
100GitHub Pages server will automatically redirect HTTP requests to the top-level
101domain if your =CNAME= file contains =example.com= but =www.example.com= is
102requested.
107103
108The same is valid if your =CNAME= file contains =www.example.com= but
109the header =Host= in the =HTTP= request contains =example.com=.
104The same is valid if your =CNAME= file contains =www.example.com= but the header
105=Host= in the =HTTP= request contains =example.com=.
110106#+end_quote
111107
112108** Why can't I add a =CNAME= record entry that accepts a top-level request (=@=) to my DNS configuration?
@@ -114,18 +110,14 @@ the header =Host= in the =HTTP= request contains =example.com=.
114110Quote from the GitHub Pages documentation:
115111
116112#+begin_quote
117Warning: Do not create a CNAME record for your custom apex domain! Doing
118so may cause issues with other services, such as email, on that domain.
113Warning: Do not create a CNAME record for your custom apex domain! Doing so may
114cause issues with other services, such as email, on that domain.
119115#+end_quote
120116
121117** References:
122118
1231. [[https://docs.github.com/en/github/working-with-github-pages/configuring-a-custom-domain-for-your-github-pages-site][Setting
124 up a custom domain with GitHub Pages]]
1252. [[https://docs.github.com/en/github/working-with-github-pages/troubleshooting-custom-domains-and-github-pages][My
126 custom domain isn't working]]
1273. [[https://serverfault.com/questions/589370/cannot-access-my-github-pages-website-by-ip-address][Cannot
128 access my GitHub Pages website by IP Address]]
1294. [[https://stackoverflow.com/questions/23375422/how-do-i-set-up-github-pages-to-redirect-dns-requests-from-a-subdomain-e-g-www][How
130 do I set up GitHub Pages to redirect DNS requests from a subdomain
131 (e.g. www) to the top-level domain (TLD, Apex record)?]]
1191. [[https://docs.github.com/en/github/working-with-github-pages/configuring-a-custom-domain-for-your-github-pages-site][Setting up a custom domain with GitHub Pages]]
1202. [[https://docs.github.com/en/github/working-with-github-pages/troubleshooting-custom-domains-and-github-pages][My custom domain isn't working]]
1213. [[https://serverfault.com/questions/589370/cannot-access-my-github-pages-website-by-ip-address][Cannot access my GitHub Pages website by IP Address]]
1224. [[https://stackoverflow.com/questions/23375422/how-do-i-set-up-github-pages-to-redirect-dns-requests-from-a-subdomain-e-g-www][How do I set up GitHub Pages to redirect DNS requests from a subdomain (e.g.
123 www) to the top-level domain (TLD, Apex record)?]]
content/blog/2020-08-29-php-auth-flow.org +40 −47
@@ -5,16 +5,15 @@
55
66* Introduction
77
8When creating websites that will allow users to create accounts, the
9developer always needs to consider the proper authentication flow for
10their app. For example, some developers will utilize an API for
11authentication, some will use OAuth, and some may just use their own
12simple database.
8When creating websites that will allow users to create accounts, the developer
9always needs to consider the proper authentication flow for their app. For
10example, some developers will utilize an API for authentication, some will use
11OAuth, and some may just use their own simple database.
1312
14For those using pre-built libraries, authentication may simply be a
15problem of copying and pasting the code from their library's
16documentation. For example, here's the code I use to authenticate users
17with the Tumblr OAuth API for my Tumblr client, Vox Populi:
13For those using pre-built libraries, authentication may simply be a problem of
14copying and pasting the code from their library's documentation. For example,
15here's the code I use to authenticate users with the Tumblr OAuth API for my
16Tumblr client, Vox Populi:
1817
1918#+begin_src php
2019// Start the session
@@ -39,24 +38,23 @@ $client = new Tumblr\API\Client(
3938);
4039#+end_src
4140
42However, developers creating authentication flows from scratch will need
43to think carefully about when to make sure a web page will check the
44user's authenticity.
41However, developers creating authentication flows from scratch will need to
42think carefully about when to make sure a web page will check the user's
43authenticity.
4544
46In this article, we're going to look at a simple authentication flow
47using a MySQL database and PHP.
45In this article, we're going to look at a simple authentication flow using a
46MySQL database and PHP.
4847
4948* Creating User Accounts
5049
51The beginning to any type of user authentication is to create a user
52account. This process can take many formats, but the simplest is to
53accept user input from a form (e.g., username and password) and send it
54over to your database. For example, here's a snippet that shows how to
55get username and password parameters that would come when a user submits
56a form to your PHP script.
50The beginning to any type of user authentication is to create a user account.
51This process can take many formats, but the simplest is to accept user input
52from a form (e.g., username and password) and send it over to your database. For
53example, here's a snippet that shows how to get username and password parameters
54that would come when a user submits a form to your PHP script.
5755
58*Note*: Ensure that your password column is large enough to hold the
59hashed value (at least 60 characters or longer).
56*Note*: Ensure that your password column is large enough to hold the hashed
57value (at least 60 characters or longer).
6058
6159#+begin_src php
6260// Get the values from the URL
@@ -95,9 +93,9 @@ $conn->close();
9593
9694** Validate Returning Users
9795
98To be able to verify that a returning user has a valid username and
99password in your database is as simple as having users fill out a form
100and comparing their inputs to your database.
96To be able to verify that a returning user has a valid username and password in
97your database is as simple as having users fill out a form and comparing their
98inputs to your database.
10199
102100#+begin_src php
103101// Query the database for username and password
@@ -114,24 +112,23 @@ if(password_verify($password_input, $hashed_password)) {
114112
115113* Storing Authentication State
116114
117Once you've created the user's account, now you're ready to initialize
118the user's session. *You will need to do this on every page you load
119while the user is logged in.* To do so, simply enter the following code
120snippet:
115Once you've created the user's account, now you're ready to initialize the
116user's session. *You will need to do this on every page you load while the user
117is logged in.* To do so, simply enter the following code snippet:
121118
122119#+begin_src php
123120session_start();
124121#+end_src
125122
126Once you've initialized the session, the next step is to store the
127session in a cookie so that you can access it later.
123Once you've initialized the session, the next step is to store the session in a
124cookie so that you can access it later.
128125
129126#+begin_src php
130127setcookie(session_name());
131128#+end_src
132129
133Now that the session name has been stored, you'll be able to check if
134there's an active session whenever you load a page.
130Now that the session name has been stored, you'll be able to check if there's an
131active session whenever you load a page.
135132
136133#+begin_src php
137134if(isset(session_name())) {
@@ -141,9 +138,9 @@ if(isset(session_name())) {
141138
142139** Removing User Authentication
143140
144The next logical step is to give your users the option to log out once
145they are done using your application. This can be tricky in PHP since a
146few of the standard ways do not always work.
141The next logical step is to give your users the option to log out once they are
142done using your application. This can be tricky in PHP since a few of the
143standard ways do not always work.
147144
148145#+begin_src php
149146// Initialize the session.
@@ -181,14 +178,10 @@ die();
181178
182179* Wrapping Up
183180
184Now you should be ready to begin your authentication programming with
185PHP. You can create user accounts, create sessions for users across
186different pages of your site, and then destroy the user data when
187they're ready to leave.
188
189For more information on this subject, I recommend reading the
190[[https://www.php.net/][PHP Documentation]]. Specifically, you may want
191to look at [[https://www.php.net/manual/en/features.http-auth.php][HTTP
192Authentication with PHP]],
193[[https://www.php.net/manual/en/book.session.php][session handling]],
194and [[https://www.php.net/manual/en/function.hash.php][hash]].
181Now you should be ready to begin your authentication programming with PHP. You
182can create user accounts, create sessions for users across different pages of
183your site, and then destroy the user data when they're ready to leave.
184
185For more information on this subject, I recommend reading the [[https://www.php.net/][PHP Documentation]].
186Specifically, you may want to look at [[https://www.php.net/manual/en/features.http-auth.php][HTTP Authentication with PHP]], [[https://www.php.net/manual/en/book.session.php][session
187handling]], and [[https://www.php.net/manual/en/function.hash.php][hash]].
content/blog/2020-09-01-visual-recognition.org +60 −71
@@ -5,38 +5,31 @@
55
66* What is IBM Watson?
77
8If you've never heard of [[https://www.ibm.com/watson][Watson]], this
9service is a suite of enterprise-ready AI services, applications, and
10tooling provided by IBM. Watson contains quite a few useful tools for
11data scientists and students, including the subject of this post today:
12visual recognition.
8If you've never heard of [[https://www.ibm.com/watson][Watson]], this service is a suite of enterprise-ready AI
9services, applications, and tooling provided by IBM. Watson contains quite a few
10useful tools for data scientists and students, including the subject of this
11post today: visual recognition.
1312
14If you'd like to view the official documentation for the Visual
15Recognition API, visit the
16[[https://cloud.ibm.com/apidocs/visual-recognition/visual-recognition-v3?code=python][API
17Docs]].
13If you'd like to view the official documentation for the Visual Recognition API,
14visit the [[https://cloud.ibm.com/apidocs/visual-recognition/visual-recognition-v3?code=python][API Docs]].
1815
1916* Prerequisites
2017
2118To be able to use Watson Visual Recognition, you'll need the following:
2219
231. Create a free account on
24 [[https://www.ibm.com/cloud/watson-studio][IBM Watson Studio]].
252. Add the [[https://www.ibm.com/cloud/watson-visual-recognition][Watson
26 Visual Recognition]] service to your IBM Watson account.
273. Get your API key and URL. To do this, first go to the
28 [[https://dataplatform.cloud.ibm.com/home2?context=cpdaas][profile
29 dashboard]] for your IBM account and click on the Watson Visual
30 Recognition service you created. This will be listed in the section
31 titled *Your services*. Then click the *Credentials* tab and open the
32 *Auto-generated credentials* dropdown. Copy your API key and URL so
33 that you can use them in the Python script later.
344. *[Optional]* While not required, you can also create the Jupyter
35 Notebook for this project right inside
36 [[https://www.ibm.com/cloud/watson-studio][Watson Studio]]. Watson
37 Studio will save your notebooks inside an organized project and allow
38 you to use their other integrated products, such as storage
39 containers, AI models, documentation, external sharing, etc.
201. Create a free account on [[https://www.ibm.com/cloud/watson-studio][IBM Watson Studio]].
212. Add the [[https://www.ibm.com/cloud/watson-visual-recognition][Watson Visual Recognition]] service to your IBM Watson account.
223. Get your API key and URL. To do this, first go to the [[https://dataplatform.cloud.ibm.com/home2?context=cpdaas][profile dashboard]] for
23 your IBM account and click on the Watson Visual Recognition service you
24 created. This will be listed in the section titled *Your services*. Then
25 click the *Credentials* tab and open the *Auto-generated credentials*
26 dropdown. Copy your API key and URL so that you can use them in the Python
27 script later.
284. *[Optional]* While not required, you can also create the Jupyter Notebook for
29 this project right inside [[https://www.ibm.com/cloud/watson-studio][Watson Studio]]. Watson Studio will save your
30 notebooks inside an organized project and allow you to use their other
31 integrated products, such as storage containers, AI models, documentation,
32 external sharing, etc.
4033
4134* Calling the IBM Watson Visual Recognition API
4235
@@ -48,8 +41,8 @@ To begin, we need to install the proper Python package for IBM Watson.
4841pip install --upgrade --user "ibm-watson>=4.5.0"
4942#+end_src
5043
51Next, we need to specify the API key, version, and URL given to us when
52we created the Watson Visual Recognition service.
44Next, we need to specify the API key, version, and URL given to us when we
45created the Watson Visual Recognition service.
5346
5447#+begin_src python
5548apikey = "<your-apikey>"
@@ -73,15 +66,15 @@ visual_recognition = VisualRecognitionV3(
7366visual_recognition.set_service_url(url)
7467#+end_src
7568
76*[Optional]* If you'd like to tell the API not to use any data to
77improve their products, set the following header.
69*[Optional]* If you'd like to tell the API not to use any data to improve their
70products, set the following header.
7871
7972#+begin_src python
8073visual_recognition.set_default_headers({'x-watson-learning-opt-out': "true"})
8174#+end_src
8275
83Now we have our API all set and ready to go. For this example, I'm going
84to include a =dict= of photos to load as we test out the API.
76Now we have our API all set and ready to go. For this example, I'm going to
77include a =dict= of photos to load as we test out the API.
8578
8679#+begin_src python
8780data = [
@@ -112,14 +105,14 @@ data = [
112105]
113106#+end_src
114107
115Now that we've set up our libraries and have the photos ready, let's
116create a loop to call the API for each image. The code below shows a
117loop that calls the URL of each image and sends it to the API,
118requesting results with at least 60% confidence. The results are output
119to the console with dotted lines separating each section.
108Now that we've set up our libraries and have the photos ready, let's create a
109loop to call the API for each image. The code below shows a loop that calls the
110URL of each image and sends it to the API, requesting results with at least 60%
111confidence. The results are output to the console with dotted lines separating
112each section.
120113
121In the case of an API error, the codes and explanations are output to
122the console.
114In the case of an API error, the codes and explanations are output to the
115console.
123116
124117#+begin_src python
125118from ibm_watson import ApiException
@@ -146,23 +139,22 @@ except ApiException as ex:
146139
147140* The Results
148141
149Here we can see the full result set of our function above. If you view
150each of the URLs that we sent to the API, you'll be able to see that it
151was remarkably accurate. To be fair, these are clear high-resolution,
152clear photos shot with a professional camera. In reality, you will most
153likely be processing images that are lower quality and may have a lot of
154noise in the photo.
142Here we can see the full result set of our function above. If you view each of
143the URLs that we sent to the API, you'll be able to see that it was remarkably
144accurate. To be fair, these are clear high-resolution, clear photos shot with a
145professional camera. In reality, you will most likely be processing images that
146are lower quality and may have a lot of noise in the photo.
155147
156However, we can clearly see the benefit of being able to call this API
157instead of attempting to write our own image recognition function. Each
158of the classifications returned was a fair description of the image.
148However, we can clearly see the benefit of being able to call this API instead
149of attempting to write our own image recognition function. Each of the
150classifications returned was a fair description of the image.
159151
160If you wanted to restrict the results to those that are at least 90%
161confident or greater, you would simply adjust the =threshold= in the
152If you wanted to restrict the results to those that are at least 90% confident
153or greater, you would simply adjust the =threshold= in the
162154=visual_recognition.classify()= function.
163155
164When your program runs, it should show the output below for each photo
165you provide.
156When your program runs, it should show the output below for each photo you
157provide.
166158
167159#+begin_src txt
168160----------------------------------------------------------------
@@ -181,22 +173,19 @@ greenishness color ( 0.975 )
181173
182174* Discussion
183175
184Now, this was a very minimal implementation of the API. We simply
185supplied some images and looked to see how accurate the results were.
186However, you could implement this type of API into many machine learning
187(ML) models.
188
189For example, you could be working for a company that scans their
190warehouses or inventory using drones. Would you want to pay employees to
191sit there and watch drone footage all day in order to identify or count
192things in the video? Probably not. Instead, you could use a
193classification system similar to this one in order to train your machine
194learning model to correctly identify items that the drones show through
195video. More specifically, you could have your machine learning model
196watch a drone fly over a field of sheep in order to count how many sheep
197are living in that field.
198
199There are many ways to implement machine learning functionality, but
200hopefully this post helped inspire some deeper thought about the tools
201that can help propel us further into the future of machine learning and
202AI.
176Now, this was a very minimal implementation of the API. We simply supplied some
177images and looked to see how accurate the results were. However, you could
178implement this type of API into many machine learning (ML) models.
179
180For example, you could be working for a company that scans their warehouses or
181inventory using drones. Would you want to pay employees to sit there and watch
182drone footage all day in order to identify or count things in the video?
183Probably not. Instead, you could use a classification system similar to this one
184in order to train your machine learning model to correctly identify items that
185the drones show through video. More specifically, you could have your machine
186learning model watch a drone fly over a field of sheep in order to count how
187many sheep are living in that field.
188
189There are many ways to implement machine learning functionality, but hopefully
190this post helped inspire some deeper thought about the tools that can help
191propel us further into the future of machine learning and AI.
content/blog/2020-09-22-internal-audit.org +161 −176
@@ -5,69 +5,67 @@
55
66* Definitions
77
8One of the many reasons that Internal Audit needs such thorough
9explaining to non-auditors is that Internal Audit can serve many
10purposes, depending on the organization's size and needs. However, the
11Institute of Internal Auditors (IIA) defines Internal Auditing as:
8One of the many reasons that Internal Audit needs such thorough explaining to
9non-auditors is that Internal Audit can serve many purposes, depending on the
10organization's size and needs. However, the Institute of Internal Auditors (IIA)
11defines Internal Auditing as:
1212
1313#+begin_quote
14Internal auditing is an independent, objective assurance and consulting
15activity designed to add value and improve an organization's operations.
16It helps an organization accomplish its objectives by bringing a
17systematic, disciplined approach to evaluate and improve the
18effectiveness of risk management, control, and governance processes.
14Internal auditing is an independent, objective assurance and consulting activity
15designed to add value and improve an organization's operations. It helps an
16organization accomplish its objectives by bringing a systematic, disciplined
17approach to evaluate and improve the effectiveness of risk management, control,
18and governance processes.
1919#+end_quote
2020
21However, this definition uses quite a few terms that aren't clear unless
22the reader already has a solid understanding of the auditing profession.
23To further explain, the following is a list of definitions that can help
24supplement understanding of internal auditing.
21However, this definition uses quite a few terms that aren't clear unless the
22reader already has a solid understanding of the auditing profession. To further
23explain, the following is a list of definitions that can help supplement
24understanding of internal auditing.
2525
2626** Independent
2727
28Independence is the freedom from conditions that threaten the ability of
29the internal audit activity to carry out internal audit responsibilities
30in an unbiased manner. To achieve the degree of independence necessary
31to effectively carry out the responsibilities of the internal audit
32activity, the chief audit executive has direct and unrestricted access
33to senior management and the board. This can be achieved through a
34dual-reporting relationship. Threats to independence must be managed at
35the individual auditor, engagement, functional, and organizational
36levels.
28Independence is the freedom from conditions that threaten the ability of the
29internal audit activity to carry out internal audit responsibilities in an
30unbiased manner. To achieve the degree of independence necessary to effectively
31carry out the responsibilities of the internal audit activity, the chief audit
32executive has direct and unrestricted access to senior management and the board.
33This can be achieved through a dual-reporting relationship. Threats to
34independence must be managed at the individual auditor, engagement, functional,
35and organizational levels.
3736
3837** Objective
3938
40Objectivity is an unbiased mental attitude that allows internal auditors
41to perform engagements in such a manner that they believe in their work
42product and that no quality compromises are made. Objectivity requires
43that internal auditors do not subordinate their judgment on audit
44matters to others. Threats to objectivity must be managed at the
45individual auditor, engagement, functional, and organizational levels.
39Objectivity is an unbiased mental attitude that allows internal auditors to
40perform engagements in such a manner that they believe in their work product and
41that no quality compromises are made. Objectivity requires that internal
42auditors do not subordinate their judgment on audit matters to others. Threats
43to objectivity must be managed at the individual auditor, engagement,
44functional, and organizational levels.
4645
4746** Assurance
4847
49Assurance services involve the internal auditor's objective assessment
50of evidence to provide opinions or conclusions regarding an entity,
51operation, function, process, system, or other subject matters. The
52internal auditor determines the nature and scope of an assurance
53engagement. Generally, three parties are participants in assurance
54services: (1) the person or group directly involved with the entity,
55operation, function, process, system, or other subject
48Assurance services involve the internal auditor's objective assessment of
49evidence to provide opinions or conclusions regarding an entity, operation,
50function, process, system, or other subject matters. The internal auditor
51determines the nature and scope of an assurance engagement. Generally, three
52parties are participants in assurance services: (1) the person or group directly
53involved with the entity, operation, function, process, system, or other subject
5654
57- (the process owner), (2) the person or group making the assessment -
58 (the internal auditor), and (3) the person or group using the
59 assessment - (the user).
55- (the process owner), (2) the person or group making the assessment - (the
56 internal auditor), and (3) the person or group using the assessment - (the
57 user).
6058
6159** Consulting
6260
63Consulting services are advisory in nature and are generally performed
64at the specific request of an engagement client. The nature and scope of
65the consulting engagement are subject to agreement with the engagement
66client. Consulting services generally involve two parties: (1) the
67person or group offering the advice (the internal auditor), and (2) the
68person or group seeking and receiving the advice (the engagement
69client). When performing consulting services, the internal auditor
70should maintain objectivity and not assume management responsibility.
61Consulting services are advisory in nature and are generally performed at the
62specific request of an engagement client. The nature and scope of the consulting
63engagement are subject to agreement with the engagement client. Consulting
64services generally involve two parties: (1) the person or group offering the
65advice (the internal auditor), and (2) the person or group seeking and receiving
66the advice (the engagement client). When performing consulting services, the
67internal auditor should maintain objectivity and not assume management
68responsibility.
7169
7270** Governance, Risk Management, & Compliance (GRC)
7371
@@ -77,172 +75,159 @@ reliably achieve objectives, address uncertainty and act with integrity.
7775* Audit Charter & Standards
7876
7977First, it's important to note that not every organization needs internal
80auditors. In fact, it's unwise for an organization to hire internal
81auditors unless they have regulatory requirements for auditing and have
82the capital to support the department. Internal audit is a cost center
83that can only affect revenue indirectly.
84
85Once an organization determines the need for internal assurance
86services, they will hire a Chief Audit Executive and create the audit
87charter. This charter is a document, approved by the company's governing
88body, that will define internal audit's purpose, authority,
89responsibility, and position within the organization. Fortunately, the
90IIA has model charters available to IIA members for those developing or
91improving their charter.
92
93Beyond the charter and organizational documents, internal auditors
94follow a few different standards in order to perform their job. First is
95the International Professional Practices Framework (IPPF) by the IIA,
96which is the model of standards for internal auditing. In addition,
97ISACA's Information Technology Assurance Framework (ITAF) helps guide
98auditors in reference to information technology (IT) compliance and
99assurance. Finally, additional standards such as FASB, GAAP, and
100industry-specific standards are used when performing internal audit
101work.
78auditors. In fact, it's unwise for an organization to hire internal auditors
79unless they have regulatory requirements for auditing and have the capital to
80support the department. Internal audit is a cost center that can only affect
81revenue indirectly.
82
83Once an organization determines the need for internal assurance services, they
84will hire a Chief Audit Executive and create the audit charter. This charter is
85a document, approved by the company's governing body, that will define internal
86audit's purpose, authority, responsibility, and position within the
87organization. Fortunately, the IIA has model charters available to IIA members
88for those developing or improving their charter.
89
90Beyond the charter and organizational documents, internal auditors follow a few
91different standards in order to perform their job. First is the International
92Professional Practices Framework (IPPF) by the IIA, which is the model of
93standards for internal auditing. In addition, ISACA's Information Technology
94Assurance Framework (ITAF) helps guide auditors in reference to information
95technology (IT) compliance and assurance. Finally, additional standards such as
96FASB, GAAP, and industry-specific standards are used when performing internal
97audit work.
10298
10399* Three Lines of Defense
104100
105[[https://theiia.org][The IIA]] released the original Three Lines of
106Defense model in 2013, but have released an updated version in 2020.
101[[https://theiia.org][The IIA]] released the original Three Lines of Defense model in 2013, but have
102released an updated version in 2020.
107103
108I won't go into depth about the changes made to the model in this
109article. Instead, let's take a look at the most current model.
104I won't go into depth about the changes made to the model in this article.
105Instead, let's take a look at the most current model.
110106
111107The updated model forgets the strict idea of areas performing their own
112functions or line of defense. Instead of talking about management, risk,
113and internal audit as 1-2-3, the new model creates a more fluid and
114cooperative model.
115
116Looking at this model from an auditing perspective shows us that
117auditors will need to align, communicate, and collaborate with
118management, including business area managers and chief officers, as well
119as reporting to the governing body. The governing body will instruct
120internal audit /functionally/ on their goals and track their progress
121periodically.
122
123However, the internal audit department will report /administratively/ to
124a chief officer in the company for the purposes of collaboration,
125direction, and assistance with the business. Note that in most
126situations, the governing body is the audit committee on the company's
127board of directors.
128
129The result of this structure is that internal audit is an independent
130and objective function that can provide assurance over the topics they
131audit.
108functions or line of defense. Instead of talking about management, risk, and
109internal audit as 1-2-3, the new model creates a more fluid and cooperative
110model.
111
112Looking at this model from an auditing perspective shows us that auditors will
113need to align, communicate, and collaborate with management, including business
114area managers and chief officers, as well as reporting to the governing body.
115The governing body will instruct internal audit /functionally/ on their goals
116and track their progress periodically.
117
118However, the internal audit department will report /administratively/ to a chief
119officer in the company for the purposes of collaboration, direction, and
120assistance with the business. Note that in most situations, the governing body
121is the audit committee on the company's board of directors.
122
123The result of this structure is that internal audit is an independent and
124objective function that can provide assurance over the topics they audit.
132125
133126* Audit Process
134127
135A normal audit will generally follow the same process, regardless of the
136topic. However, certain special projects or abnormal business areas may
137call for changes to the audit process. The audit process is not set in
138stone, it's simply a set of best practices so that audits can be
139performed consistently.
128A normal audit will generally follow the same process, regardless of the topic.
129However, certain special projects or abnormal business areas may call for
130changes to the audit process. The audit process is not set in stone, it's simply
131a set of best practices so that audits can be performed consistently.
140132
141While different organizations may tweak the process, it will generally
142follow this flow:
133While different organizations may tweak the process, it will generally follow
134this flow:
143135
144136** 1. Risk Assessment
145137
146138The risk assessment part of the process has historically been performed
147annually, but many organizations have moved to performing this process
148much more frequently. In fact, some organizations are moving to an agile
149approach that can take new risks into the risk assessment and
150re-prioritize risk areas on-the-go. To perform a risk assessment,
151leaders in internal audit will research industry risks, consult with
152business leaders around the company, and perform analyses on company
153data.
139annually, but many organizations have moved to performing this process much more
140frequently. In fact, some organizations are moving to an agile approach that can
141take new risks into the risk assessment and re-prioritize risk areas on-the-go.
142To perform a risk assessment, leaders in internal audit will research industry
143risks, consult with business leaders around the company, and perform analyses on
144company data.
154145
155146Once a risk assessment has been documented, the audit department has a
156prioritized list of risks that can be audited. This is usually in the
157form of auditable entities, such as business areas or departments.
147prioritized list of risks that can be audited. This is usually in the form of
148auditable entities, such as business areas or departments.
158149
159150** 2. Planning
160151
161During the planning phase of an audit, auditors will meet with the
162business area to discuss the various processes, controls, and risks
163applicable to the business. This helps the auditors determine the scope
164limits for the audit, as well as timing and subject-matter experts.
165Certain documents will be created in this phase that will be used to
166keep the audit on-track an in-scope as it goes forward.
152During the planning phase of an audit, auditors will meet with the business area
153to discuss the various processes, controls, and risks applicable to the
154business. This helps the auditors determine the scope limits for the audit, as
155well as timing and subject-matter experts. Certain documents will be created in
156this phase that will be used to keep the audit on-track an in-scope as it goes
157forward.
167158
168159** 3. Testing
169160
170The testing phase, also known as fieldwork or execution, is where
171internal auditors will take the information they've discovered and test
172it against regulations, industry standards, company rules, best
173practices, as well as validating that any processes are complete and
174accurate. For example, an audit of HR would most likely examine
175processes such as employee on-boarding, employee termination, security
176of personally identifiable information (PII), or the IT systems involved
177in these processes. Company standards would be examined and compared
178against how the processes are actually being performed day-to-day, as
179well as compared against regulations such as the Equal Employment
180Opportunity (EEO), American with Disabilities Act, and National Labor
181Relations Act.
161The testing phase, also known as fieldwork or execution, is where internal
162auditors will take the information they've discovered and test it against
163regulations, industry standards, company rules, best practices, as well as
164validating that any processes are complete and accurate. For example, an audit
165of HR would most likely examine processes such as employee on-boarding, employee
166termination, security of personally identifiable information (PII), or the IT
167systems involved in these processes. Company standards would be examined and
168compared against how the processes are actually being performed day-to-day, as
169well as compared against regulations such as the Equal Employment Opportunity
170(EEO), American with Disabilities Act, and National Labor Relations Act.
182171
183172** 4. Reporting
184173
185Once all the tests have been completed, the audit will enter the
186reporting phase. This is when the audit team will conclude on the
187evidence they've collected, interviews they've held, and any opinions
188they've formed on the controls in place. A summary of the audit
189findings, conclusions, and specific recommendations are officially
190communicated to the client through a draft report. Clients have the
191opportunity to respond to the report and submit an action plan and time
192frame. These responses become part of the final report which is
193distributed to the appropriate level of administration.
174Once all the tests have been completed, the audit will enter the reporting
175phase. This is when the audit team will conclude on the evidence they've
176collected, interviews they've held, and any opinions they've formed on the
177controls in place. A summary of the audit findings, conclusions, and specific
178recommendations are officially communicated to the client through a draft
179report. Clients have the opportunity to respond to the report and submit an
180action plan and time frame. These responses become part of the final report
181which is distributed to the appropriate level of administration.
194182
195183** 5. Follow-Up
196184
197After audits have been completed and management has formed action plans
198and time frames for audit issues, internal audit will follow up once
199that due date has arrived. In most cases, the follow-up will simply
200consist of a meeting to discuss how the action plan has been completed
201and to request documentation to prove it.
185After audits have been completed and management has formed action plans and time
186frames for audit issues, internal audit will follow up once that due date has
187arrived. In most cases, the follow-up will simply consist of a meeting to
188discuss how the action plan has been completed and to request documentation to
189prove it.
202190
203191* Audit Department Structure
204192
205193While an internal audit department is most often thought of as a team of
206194full-time employees, there are actually many different ways in which a
207department can be structured. As the world becomes more digital and
208fast-paced, outsourcing has become a more attractive option for some
209organizations. Internal audit can be fully outsourced or partially
210outsourced, allowing for flexibility in cases where turnover is high.
211
212In addition, departments can implement a rotational model. This allows
213for interested employees around the organization to rotate into the
214internal audit department for a period of time, allowing them to obtain
215knowledge of risks and controls and allowing the internal audit team to
216obtain more business area knowledge. This program is popular in very
217large organizations, but organizations tend to rotate lower-level audit
218staff instead of managers. This helps prevent any significant knowledge
219loss as auditors rotate out to business areas.
195department can be structured. As the world becomes more digital and fast-paced,
196outsourcing has become a more attractive option for some organizations. Internal
197audit can be fully outsourced or partially outsourced, allowing for flexibility
198in cases where turnover is high.
199
200In addition, departments can implement a rotational model. This allows for
201interested employees around the organization to rotate into the internal audit
202department for a period of time, allowing them to obtain knowledge of risks and
203controls and allowing the internal audit team to obtain more business area
204knowledge. This program is popular in very large organizations, but
205organizations tend to rotate lower-level audit staff instead of managers. This
206helps prevent any significant knowledge loss as auditors rotate out to business
207areas.
220208
221209* Consulting
222210
223Consulting is not an easy task at any organization, especially for a
224department that can have negative perceptions within the organization as
225the "compliance police." However, once an internal audit department has
226delivered value to organization, adding consulting to their suite of
227services is a smart move. In most cases, Internal Audit can insert
228themselves into a consulting role without affecting the process of
229project management at the company. This means that internal audit can
230add objective assurance and opinions to business areas as they develop
231new processes, instead of coming in periodically to audit an area and
232file issues that could have been fixed at the beginning.
211Consulting is not an easy task at any organization, especially for a department
212that can have negative perceptions within the organization as the "compliance
213police." However, once an internal audit department has delivered value to
214organization, adding consulting to their suite of services is a smart move. In
215most cases, Internal Audit can insert themselves into a consulting role without
216affecting the process of project management at the company. This means that
217internal audit can add objective assurance and opinions to business areas as
218they develop new processes, instead of coming in periodically to audit an area
219and file issues that could have been fixed at the beginning.
233220
234221* Data Science & Data Analytics
235222
236One major piece of the internal audit function in the modern world is
237data science. While the process is data science, most auditors will
238refer to anything in this realm as data analytics. Hot topics such as
239robotic process automation (RPA), machine learning (ML), and data mining
240have taken over the auditing world in recent years. These technologies
241have been immensely helpful with increasing the effectiveness and
242efficiency of auditors.
243
244For example, mundane and repetitive tasks can be automated in order for
245auditors to make more room in their schedules for labor-intensive work.
246Further, auditors will need to adapt technologies like machine learning
247in order to extract more value from the data they're using to form
248conclusions.
223One major piece of the internal audit function in the modern world is data
224science. While the process is data science, most auditors will refer to anything
225in this realm as data analytics. Hot topics such as robotic process automation
226(RPA), machine learning (ML), and data mining have taken over the auditing world
227in recent years. These technologies have been immensely helpful with increasing
228the effectiveness and efficiency of auditors.
229
230For example, mundane and repetitive tasks can be automated in order for auditors
231to make more room in their schedules for labor-intensive work. Further, auditors
232will need to adapt technologies like machine learning in order to extract more
233value from the data they're using to form conclusions.
content/blog/2020-09-25-happiness-map.org +32 −37
@@ -5,10 +5,9 @@
55
66* Background Information
77
8The dataset (obtained from
9[[https://www.kaggle.com/unsdsn/world-happiness][Kaggle]]) used in this
10article contains a list of countries around the world, their happiness
11rankings and scores, as well as other national scoring measures.
8The dataset (obtained from [[https://www.kaggle.com/unsdsn/world-happiness][Kaggle]]) used in this article contains a list of
9countries around the world, their happiness rankings and scores, as well as
10other national scoring measures.
1211
1312Fields include:
1413
@@ -21,8 +20,8 @@ Fields include:
2120- Generosity
2221- Perceptions of corruption
2322
24There are 156 records. Since there are ~195 countries in the world, we
25can see that around 40 countries will be missing from this dataset.
23There are 156 records. Since there are ~195 countries in the world, we can see
24that around 40 countries will be missing from this dataset.
2625
2726* Install Packages
2827
@@ -35,9 +34,8 @@ analysis.
3534
3635* Import the Data
3736
38We only need a couple packages to create a choropleth map. We will use
39[[https://python-visualization.github.io/folium/][Folium]], which
40provides map visualizations in Python. We will also use geopandas and
37We only need a couple packages to create a choropleth map. We will use [[https://python-visualization.github.io/folium/][Folium]],
38which provides map visualizations in Python. We will also use geopandas and
4139pandas to wrangle our data before we put it on a map.
4240
4341#+begin_src python
@@ -47,14 +45,14 @@ import geopandas as gpd
4745import pandas as pd
4846#+end_src
4947
50To get anything to show up on a map, we need a file that will specify
51the boundaries of each country. Luckily, GeoJSON files exist (for free!)
52on the internet. To get the boundaries of every country in the world, we
53will use the GeoJSON link shown below.
48To get anything to show up on a map, we need a file that will specify the
49boundaries of each country. Luckily, GeoJSON files exist (for free!) on the
50internet. To get the boundaries of every country in the world, we will use the
51GeoJSON link shown below.
5452
55GeoPandas will take this data and load it into a dataframe so that we
56can easily match it to the data we're trying to analyze. Let's look at
57the GeoJSON dataframe:
53GeoPandas will take this data and load it into a dataframe so that we can easily
54match it to the data we're trying to analyze. Let's look at the GeoJSON
55dataframe:
5856
5957#+begin_src python
6058# Load the GeoJSON data with geopandas
@@ -62,9 +60,9 @@ geo_data = gpd.read_file('https://raw.githubusercontent.com/datasets/geo-countri
6260geo_data.head()
6361#+end_src
6462
65Next, let's load the data from the Kaggle dataset. I've downloaded this
66file, so update the file path if you have it somewhere else. After
67loading, let's take a look at this dataframe:
63Next, let's load the data from the Kaggle dataset. I've downloaded this file, so
64update the file path if you have it somewhere else. After loading, let's take a
65look at this dataframe:
6866
6967#+begin_src python
7068# Load the world happiness data with pandas
@@ -74,12 +72,11 @@ happy_data.head()
7472
7573* Clean the Data
7674
77Some countries need to be renamed, or they will be lost when you merge
78the happiness and GeoJSON dataframes. This is something I discovered
79when the map below showed empty countries. I searched both data frames
80for the missing countries to see the naming differences. Any countries
81that do not have records in the =happy_data= df will not show up on the
82map.
75Some countries need to be renamed, or they will be lost when you merge the
76happiness and GeoJSON dataframes. This is something I discovered when the map
77below showed empty countries. I searched both data frames for the missing
78countries to see the naming differences. Any countries that do not have records
79in the =happy_data= df will not show up on the map.
8380
8481#+begin_src python
8582# Rename some countries to match our GeoJSON data
@@ -104,11 +101,11 @@ happy_data.at[democratic_congo_index, 'Country or region'] = 'Democratic Republi
104101* Merge the Data
105102
106103Now that we have clean data, we need to merge the GeoJSON data with the
107happiness data. Since we've stored them both in dataframes, we just need
108to call the =.merge()= function.
104happiness data. Since we've stored them both in dataframes, we just need to call
105the =.merge()= function.
109106
110We will also rename a couple columns, just so that they're a little
111easier to use when we create the map.
107We will also rename a couple columns, just so that they're a little easier to
108use when we create the map.
112109
113110#+begin_src python
114111# Merge the two previous dataframes into a single geopandas dataframe
@@ -122,10 +119,9 @@ merged_df = merged_df.rename(columns = {'Country or region':'Country'})
122119* Create the Map
123120
124121The data is finally ready to be added to a map. The code below shows the
125simplest way to find the center of the map and create a Folium map
126object. The important part is to remember to reference the merged
127dataframe for our GeoJSON data and value data. The columns specify which
128geo data and value data to use.
122simplest way to find the center of the map and create a Folium map object. The
123important part is to remember to reference the merged dataframe for our GeoJSON
124data and value data. The columns specify which geo data and value data to use.
129125
130126#+begin_src python
131127# Assign centroids to map
@@ -157,10 +153,9 @@ Let's look at the resulting map.
157153
158154* Create a Tooltip on Hover
159155
160Now that we have a map set up, we could stop. However, I want to add a
161tooltip so that I can see more information about each country. The
162=tooltip_data= code below will show a popup on hover with all the data
163fields shown.
156Now that we have a map set up, we could stop. However, I want to add a tooltip
157so that I can see more information about each country. The =tooltip_data= code
158below will show a popup on hover with all the data fields shown.
164159
165160#+begin_src python
166161 # Adding labels to map
content/blog/2020-10-12-mediocrity.org +84 −95
@@ -5,120 +5,109 @@
55
66* Perfect is the Enemy of Good
77
8As the saying goes, "the best is the enemy of the good." As we strive
9for perfection, we often fail to realize the implications of such an
10undertaking. Attempting to reach perfection is often unrealistic. Even
11worse, it can get in the way of achieving a good outcome. In certain
12situations, we try so hard to achieve the ideal solution that we have
13burned the bridges that would have allowed us to reach a lesser yet
14still superb solution.
15
16Philosophers throughout history have inspected this plight from many
17viewpoints. Greek mythology speaks of the
18[[https://en.wikipedia.org/wiki/Golden_mean_(philosophy)][golden mean]],
19which uses the story of Icarus to illustrate that sometimes "the middle
20course" is the best solution. In this story, Daedalus, a famous artist
21of his time, built feathered wings for himself and his son so that they
22might escape the clutches of King Minos. Daedalus warns his beloved son
23whom he loved so much to "fly the middle course", between the sea spray
24and the sun's heat. Icarus did not heed his father; he flew up and up
25until the sun melted the wax off his wings. For not heeding the middle
26course, he fell into the sea and drowned.
27
28More recently, management scholars have explored the
29[[https://en.wikipedia.org/wiki/Pareto_principle][Pareto principle]] and
30found that as we increase the frequency of something, or strive to
31perform actions to achieve some form of perfection, we run into
32[[https://en.wikipedia.org/wiki/Diminishing_returns][diminishing
33returns]].
34
35Even further, Harold Demsetz is noted as coining the term
36[[https://en.wikipedia.org/wiki/Nirvana_fallacy][the Nirvana fallacy]]
37in 1969, which shows the fallacy of comparing actual things with
38unrealistic, idealized alternatives. This is another trap that we may
39fall into, where we are constantly thinking of the ultimate solutions to
40problems, when something more realistic needs to be considered.
8As the saying goes, "the best is the enemy of the good." As we strive for
9perfection, we often fail to realize the implications of such an undertaking.
10Attempting to reach perfection is often unrealistic. Even worse, it can get in
11the way of achieving a good outcome. In certain situations, we try so hard to
12achieve the ideal solution that we have burned the bridges that would have
13allowed us to reach a lesser yet still superb solution.
14
15Philosophers throughout history have inspected this plight from many viewpoints.
16Greek mythology speaks of the [[https://en.wikipedia.org/wiki/Golden_mean_(philosophy)][golden mean]], which uses the story of Icarus to
17illustrate that sometimes "the middle course" is the best solution. In this
18story, Daedalus, a famous artist of his time, built feathered wings for himself
19and his son so that they might escape the clutches of King Minos. Daedalus warns
20his beloved son whom he loved so much to "fly the middle course", between the
21sea spray and the sun's heat. Icarus did not heed his father; he flew up and up
22until the sun melted the wax off his wings. For not heeding the middle course,
23he fell into the sea and drowned.
24
25More recently, management scholars have explored the [[https://en.wikipedia.org/wiki/Pareto_principle][Pareto principle]] and found
26that as we increase the frequency of something, or strive to perform actions to
27achieve some form of perfection, we run into [[https://en.wikipedia.org/wiki/Diminishing_returns][diminishing returns]].
28
29Even further, Harold Demsetz is noted as coining the term [[https://en.wikipedia.org/wiki/Nirvana_fallacy][the Nirvana fallacy]] in
301969, which shows the fallacy of comparing actual things with unrealistic,
31idealized alternatives. This is another trap that we may fall into, where we are
32constantly thinking of the ultimate solutions to problems, when something more
33realistic needs to be considered.
4134
4235Over and over throughout history, we've found that perfection is often
43unrealistic and unachievable. However, we push ourselves and our peers
44to "give 100%" or "go the extra mile," while it may be that the better
45course is to give a valuable level of effort while considering the
46effects of further effort on the outcome. Working harder does not always
47help us achieve loftier goals.
48
49This has presented itself to me most recently during my time studying at
50my university. I was anxious and feeling the stresses of my courses,
51career, and personal life for quite a while, which was greatly affecting
52how well I was doing at school and my level of effort at work. One day,
53I happened to be talking to my father when he said something simple that
54hit home:
36unrealistic and unachievable. However, we push ourselves and our peers to "give
37100%" or "go the extra mile," while it may be that the better course is to give
38a valuable level of effort while considering the effects of further effort on
39the outcome. Working harder does not always help us achieve loftier goals.
40
41This has presented itself to me most recently during my time studying at my
42university. I was anxious and feeling the stresses of my courses, career, and
43personal life for quite a while, which was greatly affecting how well I was
44doing at school and my level of effort at work. One day, I happened to be
45talking to my father when he said something simple that hit home:
5546
5647#+begin_quote
57All you can do is show up and do your best. Worrying about the outcomes
58won't affect the outcome itself.
48All you can do is show up and do your best. Worrying about the outcomes won't
49affect the outcome itself.
5950#+end_quote
6051
6152The thought was extremely straightforward and uncomplicated, yet it was
62something that I had lost sight of during my stress-filled years at
63school. Ever since then, I've found myself pausing and remembering that
64quote every time I get anxious or stressed. It helps to stop and think
65"Can I do anything to affect the outcome, or am I simply worrying over
66something I can't change?"
53something that I had lost sight of during my stress-filled years at school. Ever
54since then, I've found myself pausing and remembering that quote every time I
55get anxious or stressed. It helps to stop and think "Can I do anything to affect
56the outcome, or am I simply worrying over something I can't change?"
6757
6858* When Mediocrity Isn't Enough
6959
70One problem with the philosophies presented in this post is that they
71are implemented far too often in situations where mediocrity simply
72isn't adequate. For example, let's take a look at digital user data,
73specifically personally-identifiable information (PII). As a
74cybersecurity auditor in the United States, I have found that most
75companies are concerned more with compliance than any actual safeguards
76over the privacy or protection of user data. Other than companies who
77have built their reputation on privacy and security, most companies will
78use [[https://en.wikipedia.org/wiki/Satisficing][satisficing]] as their
79primary decision-making strategy around user data.
60One problem with the philosophies presented in this post is that they are
61implemented far too often in situations where mediocrity simply isn't adequate.
62For example, let's take a look at digital user data, specifically
63personally-identifiable information (PII). As a cybersecurity auditor in the
64United States, I have found that most companies are concerned more with
65compliance than any actual safeguards over the privacy or protection of user
66data. Other than companies who have built their reputation on privacy and
67security, most companies will use [[https://en.wikipedia.org/wiki/Satisficing][satisficing]] as their primary decision-making
68strategy around user data.
8069
8170#+begin_quote
82Satisficing is a decision-making strategy or cognitive heuristic that
83entails searching through the available alternatives until an
84acceptability threshold is met.
71Satisficing is a decision-making strategy or cognitive heuristic that entails
72searching through the available alternatives until an acceptability threshold is
73met.
8574#+end_quote
8675
87This means that each decision will be met with certain possible
88solutions until one of the solutions meets their minimum acceptable
89standards. For companies that deal with user data, the
90minimum-acceptable standards come from three areas:
76This means that each decision will be met with certain possible solutions until
77one of the solutions meets their minimum acceptable standards. For companies
78that deal with user data, the minimum-acceptable standards come from three
79areas:
9180
92811. Laws and regulations
93822. Competitive pressure
94833. Risk of monetary or reputation loss
9584
96Working with project management or auditing, the primary concern here is
97the risk of legal ramifications. Since the primary risk comes from laws
98and regulations, companies will require that any project that involves
99user data must follow all the rules of those laws so that the company
100can protect itself from fines or other penalties.
101
102Following this, companies will consider best practices in order to place
103itself in a competitive position (e.g. Google vs. Apple) and review any
104recent or ongoing litigation against companies regarding user data. In a
105perfect company, management would then consider the ethical
106responsibilities of their organization and discuss their
107responsibilities over things like personally-identifiable information.
108
109However, as we mentioned above, most companies follow the idea of
110satisficing, which states that they have met the minimum acceptable
111standards and can now move on to other decisions. Modern business
112culture in the United States dictates that profits are the golden
113measure of how well a company or manager is performing, so we often
114don't think about our responsibilities beyond these basic standards.
115
116Not all situations demand excellence, but I believe that applying any
117philosophy as a broad stroke across one's life can be a mistake. We must
118be able to think critically about what we are doing as we do it and ask
119ourselves a few questions. Have I done everything I can in this
120situation? Is mediocrity an acceptable outcome, or should we strive for
121perfection, even if we can't attain it?
85Working with project management or auditing, the primary concern here is the
86risk of legal ramifications. Since the primary risk comes from laws and
87regulations, companies will require that any project that involves user data
88must follow all the rules of those laws so that the company can protect itself
89from fines or other penalties.
90
91Following this, companies will consider best practices in order to place itself
92in a competitive position (e.g. Google vs. Apple) and review any recent or
93ongoing litigation against companies regarding user data. In a perfect company,
94management would then consider the ethical responsibilities of their
95organization and discuss their responsibilities over things like
96personally-identifiable information.
97
98However, as we mentioned above, most companies follow the idea of satisficing,
99which states that they have met the minimum acceptable standards and can now
100move on to other decisions. Modern business culture in the United States
101dictates that profits are the golden measure of how well a company or manager is
102performing, so we often don't think about our responsibilities beyond these
103basic standards.
104
105Not all situations demand excellence, but I believe that applying any philosophy
106as a broad stroke across one's life can be a mistake. We must be able to think
107critically about what we are doing as we do it and ask ourselves a few
108questions. Have I done everything I can in this situation? Is mediocrity an
109acceptable outcome, or should we strive for perfection, even if we can't attain
110it?
122111
123112Taking a few moments to think critically throughout our day, as we make
124113decisions, can have a tremendous effect on the outcomes we create.
content/blog/2020-12-27-website-redesign.org +71 −84
@@ -5,96 +5,83 @@
55
66* A Brief History
77
8As a form of continuous learning and entertainment, I've been running a
9handful of websites since 2016 when I took my first programming courses
10in college. I maintain one main website, the place I consider the
11official website to represent me. Under this site, I have a handful of
12subdirectories and subdomains.
13
14One of the parts I've enjoyed the most about web development is the
15aspect of designing an identity for a web page and working to find
16exciting ways to display the site's content. Inevitably, this means I've
17changed the designs for my websites more times than I could possibly
18count. Since I don't really host anything on my main webpage that's
19vital, it allows me the freedom to change things as inspiration strikes.
20
21Historically, I've relied on core utilities for spacing, components, and
22layouts from [[https://getbootstrap.com][Bootstrap]] and added custom
23CSS for fonts, accents, colors, and other items. I also tend to create
24sites with no border radius on items, visible borders, and content that
25takes up the entire screen (using whitespace inside components instead
26of whitespace around my components).
8As a form of continuous learning and entertainment, I've been running a handful
9of websites since 2016 when I took my first programming courses in college. I
10maintain one main website, the place I consider the official website to
11represent me. Under this site, I have a handful of subdirectories and
12subdomains.
13
14One of the parts I've enjoyed the most about web development is the aspect of
15designing an identity for a web page and working to find exciting ways to
16display the site's content. Inevitably, this means I've changed the designs for
17my websites more times than I could possibly count. Since I don't really host
18anything on my main webpage that's vital, it allows me the freedom to change
19things as inspiration strikes.
20
21Historically, I've relied on core utilities for spacing, components, and layouts
22from [[https://getbootstrap.com][Bootstrap]] and added custom CSS for fonts, accents, colors, and other items.
23I also tend to create sites with no border radius on items, visible borders, and
24content that takes up the entire screen (using whitespace inside components
25instead of whitespace around my components).
2726
2827* The Redesign Process
2928
30About a week ago, I found myself wishing for a new design yet again. The
31prior design was largely inspired by IBM's
32[[https://www.carbondesignsystem.com][Carbon Design System]] and relied
33on jQuery, Bootstrap, along with some compressed
34[[https://en.wikipedia.org/wiki/WebP][.webp]] images.
35
36To anyone who knows my preferences toward web design - and even in my
37personal life - it should be no surprise that I immediately started
38looking for inspiration on minimalism. While there are some decent
39minimalistic designs on sites like
40[[https://dribbble.com/search/shots/popular/web-design?q=minimalism][Dribbble]],
41people seem to mostly discuss [[https://brutalist-web.design][brutalist
42web design]] when you ask about minimalism. While brutalist web design
43doesn't have to be minimal, it often is.
44
45I suppose, in a way, I did create a brutalist website since my HTML is
46semantic and accessible, hyperlinks are colored and underlined, and all
47native browser functions like scrolling and the back button work as
48expected. However, I didn't think about brutalism while designing these
49sites.
50
51The new design followed a simple design process. I walked through the
52screens on my blog and asked myself: "Is this element necessary for a
53user?" This allowed me to first start by removing all javascript, which
54had the sole purpose of allowing users to open a collapsed navbar on
55mobile. Replacing the collapsible navbar allowed me to remove both
56jQuery and Bootstrap's javascript.
57
58Next, I removed things like author names (since I'm literally the only
59person who will ever write on this site), multiple ways to click on a
60blog post card, blog post descriptions, and the scroll-to-top button. It
61also helped to move all categories to a single page, rather than have
62each category on its own page.
63
64The final big piece to finish the
65"[[https://en.wikipedia.org/wiki/Marie_Kondo#KonMari_method][KonMari]]"-like
66part of my process was to remove Bootstrap CSS in its entirety. However,
67this meant pulling out a few very useful classes, such as =.img-fluid=
68and the default font stacks to keep in my custom CSS.
69
70After removing all the unnecessary pieces, I was finally able to
71reorganize my content and add a very small amount of custom CSS to make
72everything pretty. This took a brief amount of time, effectively just
73consisting of me converting =<div>= tags into things like =<ul>= lists
74and choosing accent colors.
29About a week ago, I found myself wishing for a new design yet again. The prior
30design was largely inspired by IBM's [[https://www.carbondesignsystem.com][Carbon Design System]] and relied on jQuery,
31Bootstrap, along with some compressed [[https://en.wikipedia.org/wiki/WebP][.webp]] images.
32
33To anyone who knows my preferences toward web design - and even in my personal
34life - it should be no surprise that I immediately started looking for
35inspiration on minimalism. While there are some decent minimalistic designs on
36sites like [[https://dribbble.com/search/shots/popular/web-design?q=minimalism][Dribbble]], people seem to mostly discuss [[https://brutalist-web.design][brutalist web design]] when you
37ask about minimalism. While brutalist web design doesn't have to be minimal, it
38often is.
39
40I suppose, in a way, I did create a brutalist website since my HTML is semantic
41and accessible, hyperlinks are colored and underlined, and all native browser
42functions like scrolling and the back button work as expected. However, I didn't
43think about brutalism while designing these sites.
44
45The new design followed a simple design process. I walked through the screens on
46my blog and asked myself: "Is this element necessary for a user?" This allowed
47me to first start by removing all javascript, which had the sole purpose of
48allowing users to open a collapsed navbar on mobile. Replacing the collapsible
49navbar allowed me to remove both jQuery and Bootstrap's javascript.
50
51Next, I removed things like author names (since I'm literally the only person
52who will ever write on this site), multiple ways to click on a blog post card,
53blog post descriptions, and the scroll-to-top button. It also helped to move all
54categories to a single page, rather than have each category on its own page.
55
56The final big piece to finish the "[[https://en.wikipedia.org/wiki/Marie_Kondo#KonMari_method][KonMari]]"-like part of my process was to
57remove Bootstrap CSS in its entirety. However, this meant pulling out a few very
58useful classes, such as =.img-fluid= and the default font stacks to keep in my
59custom CSS.
60
61After removing all the unnecessary pieces, I was finally able to reorganize my
62content and add a very small amount of custom CSS to make everything pretty.
63This took a brief amount of time, effectively just consisting of me converting
64=<div>= tags into things like =<ul>= lists and choosing accent colors.
7565
7666* The Results
7767
7868** Reflection
7969
8070So, what did all of this reorganizing do to my webpages? Well, first, my
81websites are now *ridiculously fast*. Since the prior designs were also
82minimal and didn't have many images, they measured up in Firefox's
83Network Monitor around 300 KB - 600KB. After making the changes, my main
84site is at 5 KB transferred (22 KB total), and my blog is at 6.5 KB
85transferred (13 KB total). *That means the redesigned pages are less
86than 2% the size of the old designs.*
87
88Google Lighthouse ranks the new webpage as 100 in performance,
89accessibility, and best practices, with SEO at 92 since they think tap
90targets are not sized appropriately for mobile users. First contextual
91paints of the pages are under 0.8 seconds with 0 ms of blocking time.
92However, the blog subdomain ranks at 100 for all four categories! First
93contextual paints of the blog homepage are under 1.0 seconds with 0 ms
94of blocking time, due to the fact that the CSS for my blog is within a
95separate CSS file, and the CSS for my main website is simply embedded in
96the HTML file.
97
98Now that everything is complete, I can confidently say I'm happy with
99the result and proud to look at the fastest set of websites I've created
100so far.
71websites are now *ridiculously fast*. Since the prior designs were also minimal
72and didn't have many images, they measured up in Firefox's Network Monitor
73around 300 KB - 600KB. After making the changes, my main site is at 5 KB
74transferred (22 KB total), and my blog is at 6.5 KB transferred (13 KB total).
75*That means the redesigned pages are less than 2% the size of the old designs.*
76
77Google Lighthouse ranks the new webpage as 100 in performance, accessibility,
78and best practices, with SEO at 92 since they think tap targets are not sized
79appropriately for mobile users. First contextual paints of the pages are under
800.8 seconds with 0 ms of blocking time. However, the blog subdomain ranks at 100
81for all four categories! First contextual paints of the blog homepage are under
821.0 seconds with 0 ms of blocking time, due to the fact that the CSS for my blog
83is within a separate CSS file, and the CSS for my main website is simply
84embedded in the HTML file.
85
86Now that everything is complete, I can confidently say I'm happy with the result
87and proud to look at the fastest set of websites I've created so far.
content/blog/2020-12-28-neon-drive.org +38 −49
@@ -5,80 +5,69 @@
55
66* Game Description
77
8[[https://store.steampowered.com/app/433910/Neon_Drive/][Neon Drive]]
9presents itself as a simple arcade-style game inspired by the arcade
10race games of the 1980s, yet it has managed to take up hours of my life
11without much effort. The game description, directly from the Steam page,
12is intriguing enough to entice anyone who's been looking for a good
13arcade racing game:
8[[https://store.steampowered.com/app/433910/Neon_Drive/][Neon Drive]] presents itself as a simple arcade-style game inspired by the arcade
9race games of the 1980s, yet it has managed to take up hours of my life without
10much effort. The game description, directly from the Steam page, is intriguing
11enough to entice anyone who's been looking for a good arcade racing game:
1412
1513#+begin_quote
16Neon Drive is a slick retro-futuristic arcade game that will make your
17brain melt. You've been warned. From beautiful cityscapes and ocean
18roads to exploding enemy spaceships, Neon Drive has it all.
14Neon Drive is a slick retro-futuristic arcade game that will make your brain
15melt. You've been warned. From beautiful cityscapes and ocean roads to exploding
16enemy spaceships, Neon Drive has it all.
1917#+end_quote
2018
2119* Gameplay
2220
23The game holds true to the
24[[https://en.wikipedia.org/wiki/Retrofuturism][retro-futurism]] style,
25including chrome female robots, pixelated arcade machines, and
26[[https://teddit.net/r/outrun/][outrun]] aesthetics.
21The game holds true to the [[https://en.wikipedia.org/wiki/Retrofuturism][retro-futurism]] style, including chrome female robots,
22pixelated arcade machines, and [[https://teddit.net/r/outrun/][outrun]] aesthetics.
2723
28Each level of the game is shown as a separate arcade machine. Each
29arcade machine lets you play on Normal, Hard, Insane, Practice, and Free
30Run. To beat each arcade, you must reach the end of the level without
31crashing your car into the various obstacles on the course. Basic levels
32let you move left or right to avoid blocks in the road. Later levels put
33you through other tests, such as dodging traffic or blasting asteroids.
24Each level of the game is shown as a separate arcade machine. Each arcade
25machine lets you play on Normal, Hard, Insane, Practice, and Free Run. To beat
26each arcade, you must reach the end of the level without crashing your car into
27the various obstacles on the course. Basic levels let you move left or right to
28avoid blocks in the road. Later levels put you through other tests, such as
29dodging traffic or blasting asteroids.
3430
35The game uses synthwave music to keep you on track to make the correct
36moves by timing the beats of the songs to the correct moves on the
37screen. It reminds me of the early Guitar Hero games, as well as mobile
38apps like VOEZ - repetition and staying on-beat is the only way to win.
31The game uses synthwave music to keep you on track to make the correct moves by
32timing the beats of the songs to the correct moves on the screen. It reminds me
33of the early Guitar Hero games, as well as mobile apps like VOEZ - repetition
34and staying on-beat is the only way to win.
3935
4036* In-Game Screenshots
4137
42Taking a look at the main menu, you can see that Neon Drive plays into
43every stereotype you can think of around retro-futuristic, synthwave
44arcades (in a good way).
38Taking a look at the main menu, you can see that Neon Drive plays into every
39stereotype you can think of around retro-futuristic, synthwave arcades (in a
40good way).
4541
46Once you get into the first level, we see that the choice of car fits
47right in with the stereotypical cars of the 80s, like the
48[[https://en.wikipedia.org/wiki/DMC_DeLorean][DeLorean]] or the
49[[https://en.wikipedia.org/wiki/Ferrari_F40][Ferrari F40]]. Each new
50level comes with new color schemes and cars, so you should never get
42Once you get into the first level, we see that the choice of car fits right in
43with the stereotypical cars of the 80s, like the [[https://en.wikipedia.org/wiki/DMC_DeLorean][DeLorean]] or the [[https://en.wikipedia.org/wiki/Ferrari_F40][Ferrari F40]].
44Each new level comes with new color schemes and cars, so you should never get
5145tired of the aesthetic.
5246
5347Personally, I love the orange and blue colors used in level 2:
5448
55If you're the competitive type and getting 100% on all arcade machines
56isn't enough, there are leaderboards for the regular part of the game,
57and the endurance game mode.
49If you're the competitive type and getting 100% on all arcade machines isn't
50enough, there are leaderboards for the regular part of the game, and the
51endurance game mode.
5852
5953* Other Suggestions
6054
61Neon Drive sits nicely within the well-founded cult genre of Outrun.
62Other games that I've enjoyed in this same spectrum are:
55Neon Drive sits nicely within the well-founded cult genre of Outrun. Other games
56that I've enjoyed in this same spectrum are:
6357
64- [[https://store.steampowered.com/app/233270/Far_Cry_3__Blood_Dragon/][Far
65 Cry 3: Blood Dragon]]
58- [[https://store.steampowered.com/app/233270/Far_Cry_3__Blood_Dragon/][Far Cry 3: Blood Dragon]]
6659- [[https://store.steampowered.com/app/1239690/Retrowave/][Retrowave]]
6760- [[https://store.steampowered.com/app/732810/Slipstream/][Slipstream]]
6861
69Although these games aren't necessarily in the same genre, they do have
70aspects that place them close enough to interest gamers that enjoyed
71Neon Drive:
62Although these games aren't necessarily in the same genre, they do have aspects
63that place them close enough to interest gamers that enjoyed Neon Drive:
7264
7365- [[https://store.steampowered.com/app/311800/Black_Ice/][Black Ice]]
7466- [[https://store.steampowered.com/app/746850/Cloudpunk/][Cloudpunk]]
75- [[https://store.steampowered.com/app/1222680/Need_for_Speed_Heat/][Need
76 for Speed: Heat]]
67- [[https://store.steampowered.com/app/1222680/Need_for_Speed_Heat/][Need for Speed: Heat]]
7768- [[https://store.steampowered.com/app/1019310/VirtuaVerse/][VirtuaVerse]]
7869
79Of course, if all you really care about is the arcade aspect of these
80games, you can check out the
81[[https://store.steampowered.com/app/400020/Atari_Vault/][Atari Vault]]
82or any of the other classic games sold on Steam by companies like Namco,
83Atari. For something like Nintendo, you'd have to settle for buying used
84classic consoles or delve into the world of emulation.
70Of course, if all you really care about is the arcade aspect of these games, you
71can check out the [[https://store.steampowered.com/app/400020/Atari_Vault/][Atari Vault]] or any of the other classic games sold on Steam by
72companies like Namco, Atari. For something like Nintendo, you'd have to settle
73for buying used classic consoles or delve into the world of emulation.
content/blog/2020-12-29-zork.org +44 −56
@@ -5,28 +5,24 @@
55
66* Download (Free)
77
8Before we get into the game itself, you should know that you can
9download Zork for free from Infocom's
10[[http://infocom-if.org/downloads/downloads.html][download page]]. So
11feel free to boot it up and take a ride back to the 1980s with this
12masterpiece.
8Before we get into the game itself, you should know that you can download Zork
9for free from Infocom's [[http://infocom-if.org/downloads/downloads.html][download page]]. So feel free to boot it up and take a
10ride back to the 1980s with this masterpiece.
1311
1412* Game Description
1513
16Zork is an interactive, text-based computer game originally released
17in 1980. This series, split into three separate games, introduced a
18robust and sophisticated text parser to gamers. People were largely used
19to the simple commands used in the popular game
20[[https://en.wikipedia.org/wiki/Colossal_Cave_Adventure][Colossal Cave
21Adventure]], but Zork allowed users to send more complex commands that
22included prepositions and conjunctions.
23
24Zork tracks your score as you explore the map, find tools, and collect
25trophy items (e.g., a jewel-encrusted egg). When you place your trophy
26items in the trophy case found in the Living Room area, you gain score
27points. Collecting the Twenty Treasures of Zork and placing them within
28the trophy case wins the game. However, you must explore the map, solve
29puzzles, and avoid being eaten by a grue to collect these treasures.
14Zork is an interactive, text-based computer game originally released in 1980.
15This series, split into three separate games, introduced a robust and
16sophisticated text parser to gamers. People were largely used to the simple
17commands used in the popular game [[https://en.wikipedia.org/wiki/Colossal_Cave_Adventure][Colossal Cave Adventure]], but Zork allowed
18users to send more complex commands that included prepositions and conjunctions.
19
20Zork tracks your score as you explore the map, find tools, and collect trophy
21items (e.g., a jewel-encrusted egg). When you place your trophy items in the
22trophy case found in the Living Room area, you gain score points. Collecting the
23Twenty Treasures of Zork and placing them within the trophy case wins the game.
24However, you must explore the map, solve puzzles, and avoid being eaten by a
25grue to collect these treasures.
3026
3127* The Map
3228
@@ -42,45 +38,37 @@ map.
4238
4339* In-Game Screenshots
4440
45After playing the game (for the first time ever) for several weeks
46around 2014, I was finally able to beat the game with some online help
47to find the last couple items. As I was writing this post, I installed
48the game again to grab some screenshots to show off the true glory of
49this game. As noted in
50[[https://www.filfre.net/2012/01/exploring-zork-part-1/][Jimmy Maher's
51playthrough]], the original Zork games looked quite a bit different due
52to the older hardware of computers like the Apple II and multiple bug
53fixes that Infocom pushed out after the game's initial release. My
54play-through uses the
55[[https://store.steampowered.com/app/570580/Zork_Anthology/][Zork
56Anthology]] version, which utilizes DOSBox on Windows.
57
58The first screenshot here shows the introductory information, which
59doesn't include instructions of any kind for the player. If you haven't
60played text adventures before, try to use simple commands like "go
61west," "look around," or "hit troll with elvish sword."
62
63In this second screenshot, we see the player has entered the house and
64found the trophy case in the living room. The lantern and sword in this
65room allow the player to explore dark areas and attack enemies. If you
66don't use the lantern, you won't be able to see anything in dark areas,
67and you may be eaten by a grue.
68
69Finally, we see that the player has found the first treasure: a
70jewel-encrusted egg. These treasures can be taken back to the house and
71placed in the trophy case or carried until you feel like you want to put
72things away.
41After playing the game (for the first time ever) for several weeks around 2014,
42I was finally able to beat the game with some online help to find the last
43couple items. As I was writing this post, I installed the game again to grab
44some screenshots to show off the true glory of this game. As noted in [[https://www.filfre.net/2012/01/exploring-zork-part-1/][Jimmy
45Maher's playthrough]], the original Zork games looked quite a bit different due to
46the older hardware of computers like the Apple II and multiple bug fixes that
47Infocom pushed out after the game's initial release. My play-through uses the
48[[https://store.steampowered.com/app/570580/Zork_Anthology/][Zork Anthology]] version, which utilizes DOSBox on Windows.
49
50The first screenshot here shows the introductory information, which doesn't
51include instructions of any kind for the player. If you haven't played text
52adventures before, try to use simple commands like "go west," "look around," or
53"hit troll with elvish sword."
54
55In this second screenshot, we see the player has entered the house and found the
56trophy case in the living room. The lantern and sword in this room allow the
57player to explore dark areas and attack enemies. If you don't use the lantern,
58you won't be able to see anything in dark areas, and you may be eaten by a grue.
59
60Finally, we see that the player has found the first treasure: a jewel-encrusted
61egg. These treasures can be taken back to the house and placed in the trophy
62case or carried until you feel like you want to put things away.
7363
7464* Conclusion
7565
76It's been quite a few years since I first played Zork, but I clearly
77remember the late nights and bloodshot eyes that helped me find all the
78treasures. This game is well worth the time and effort, even though the
79text-based aspect may be off-putting to gamers who didn't have to grow
80up playing games without graphics. However, I believe that the strategy
81and skills learned in early video games like Zork can actually help you,
82even when playing newer games.
66It's been quite a few years since I first played Zork, but I clearly remember
67the late nights and bloodshot eyes that helped me find all the treasures. This
68game is well worth the time and effort, even though the text-based aspect may be
69off-putting to gamers who didn't have to grow up playing games without graphics.
70However, I believe that the strategy and skills learned in early video games
71like Zork can actually help you, even when playing newer games.
8372
8473If you do decide to play Zork, you can download Zork I, II, and III from
85Infocom's [[http://infocom-if.org/downloads/downloads.html][download
86page]] for free or search the internet for an online version.
74Infocom's [[http://infocom-if.org/downloads/downloads.html][download page]] for free or search the internet for an online version.