cmc/cleberg.net

  1#+date:        [2023-06-28 Wed 00:00:00]
  2#+title:       Backblaze B2 Backups
  3#+description: Setting up Backblaze B2 for offsite backups before I moved on.
  4#+slug:        backblaze-b2
  5#+filetags:    :self-hosting:
  6
  7* Overview
  8
  9Backblaze [[https://www.backblaze.com/b2/cloud-storage.html][B2 Cloud
 10Storage]] is an inexpensive and reliable on-demand cloud storage and
 11backup solution.
 12
 13The service starts at $5/TB/month ($0.005/GB/month) with a download rate
 14of $0.01/GB/month.
 15
 16However, there are free tiers:
 17
 18- The first 10 GB of storage is free.
 19- The first 1 GB of data downloaded each day is free.
 20- Class A transactions are free.
 21- The first 2500 Class B transactions each day are free.
 22- The first 2500 Class C transactions each day are free.
 23
 24You can see which API calls fall into categories A, B, or C here:
 25[[https://www.backblaze.com/b2/b2-transactions-price.html][Pricing
 26Organized by API Calls]].
 27
 28For someone like me, who wants an offsite backup of their server's
 29=/home/= directory and various other server configs that fall under 10
 30GB total, Backblaze is a great solution from a financial perspective.
 31
 32* Create An Account
 33
 34To start with Backblaze, you'll need to
 35[[https://www.backblaze.com/b2/sign-up.html][create a free account]] -
 36no payment method is required to sign up.
 37
 38Once you have an account, you can test out the service with their web
 39GUI, their mobile app, or their CLI tool. I'm going to use the CLI tool
 40below to test a file upload and then sync an entire directory to my
 41Backblaze bucket.
 42
 43* Create a Bucket
 44
 45Before you can start uploading, you need to create a bucket. If you're
 46familiar with other object storage services, this will feel familiar. If
 47not, it's pretty simple to create one.
 48
 49As their webpage says:
 50
 51#+begin_quote
 52A bucket is a container that holds files that are uploaded into B2 Cloud
 53Storage. The bucket name must be globally unique and must have a minimum
 54of 6 characters. A limit of 100 buckets may be created per account. An
 55unlimited number of files may be uploaded into a bucket.
 56#+end_quote
 57
 58Once you click the =Create a Bucket= button on their webpage or mobile
 59app, you need to provide the following:
 60
 61- Bucket Unique Name
 62- Files in Bucket are: =Private= or =Public=
 63- Default Encryption: =Disable= or =Enable=
 64- Object Lock: =Disable= or =Enable=
 65
 66For my bucket, I created a private bucket with encryption enabled and
 67object lock disabled.
 68
 69Once your bucket is created, you can test the upload/download feature on
 70their web GUI or mobile app! At this point, you have a fully functional
 71bucket and account.
 72
 73* Linux CLI Tool
 74
 75** Installation
 76
 77To install the =b2= CLI tool, you'll need to download it from the
 78[[https://www.backblaze.com/docs/cloud-storage-command-line-tools][CLI
 79Tools]] page. I recommend copying the URL from the link that says
 80=Linux= and using wget to download it, as shown below.
 81
 82Once downloaded, make the file executable and move it to a location on
 83your =$PATH=, so that you can execute that command from anywhere on the
 84machine.
 85
 86#+begin_src sh
 87wget <b2_cli_url>
 88chmod +x b2_linux
 89mv b2_linux /usr/bin/b2
 90#+end_src
 91
 92** Log In
 93
 94The first step after installation is to log in. To do this, execute the
 95following command and provide your =<applicationKeyId>= and
 96=<applicationKey>=.
 97
 98If you don't want to provide these values in the command itself, you can
 99simply execute the base command and it will request them in an
100interactive prompt.
101
102#+begin_src sh
103# if you want to provide the keys directly:
104b2 authorize-account [<applicationKeyId>] [<applicationKey>]
105
106# or, if you don't want your keys in your shell history:
107b2 authorize-account
108#+end_src
109
110** Upload a Test File
111
112In order to test the functionality of the CLI tool, I'll start by
113uploading a single test file to the bucket I created above. We can do
114this with the =upload_file= function.
115
116The command is issued as follows:
117
118#+begin_src sh
119b2 upload_file <bucket_name> <local_file> <remote_file>
120#+end_src
121
122In my situation, I executed the following command with my username.
123
124#+begin_src sh
125b2 upload_file my_unique_bucket /home/<user>/test.md test.md
126#+end_src
127
128To confirm that the file was uploaded successfully, list the files in
129your bucket:
130
131#+begin_src sh
132b2 ls <bucket_name>
133#+end_src
134
135#+begin_src txt
136test.md
137#+end_src
138
139** Sync a Directory
140
141If you have numerous files, you can use the =sync= function to perform
142functionality similar to =rsync=, where you can check what's in your
143bucket and sync anything that is new or modified.
144
145The command is issued as follows:
146
147#+begin_src sh
148b2 sync <source file location> <B2 bucket destination>
149#+end_src
150
151In my case, I can sync my user's entire home directory to my bucket
152without specifying any of the files directly:
153
154#+begin_src sh
155b2 sync /home/<user>/ "b2://<bucketName>/home/<user>"
156#+end_src
157
158* Caveats
159
160** Timing of Updates to the Web GUI
161
162When performing actions over a bucket, there is a slight delay in the
163web GUI when inspecting a bucket or its file. Note that simple actions
164such as uploading or deleting files may have a delay of a few minutes up
165to 24 hours. In my experience (<10 GB and ~20,000 files), any actions
166took only a few minutes to update across clients.
167
168** Symlinks
169
170Note that symlinks are resolved by b2, so if you have a link from
171=/home/<user>/nas-storage= that symlinks out to a =/mnt/nas-storage=
172folder that has 10TB of data, =b2= will resolve that link and start
173uploading all 10TB of data linked within the folder.
174
175If you're not sure if you have any symlinks, a symlink will look like
176this (note the =->= symbol):
177
178#+begin_src sh
179> ls -lha
180lrwxrwxrwx  1 root root   20 Jun 28 13:32 nas -> /mnt/nas-storage/
181#+end_src
182
183You can recursively find symlink in a path with the following command:
184
185#+begin_src sh
186ls -lR /path/to/search | grep '^l'
187#+end_src