Project Log for Nguyen Viet Anh

It is recommended to view this page in a wider screen.

This note all activities that Nguyen Viet Anh (@VietAnh1010) does on ezkomment project. Note that activities before 14 May were not logged properly.

Total hours spent

228

Dates (D/M)

Content

Hrs

Remarks

12/03 - 14/03

Brainstorm for ideas and finalise idea

F2F meeting with teammate
Decided the app features
Finalised proporsal for Orbital submission

15/04 - 3/5

Learn the basics of backend development

I initially started with Spring Framework, but later switched and considered Django and Express
Express was choosen because Express app can be written using Typescript, which has a better type system than Python

06/05 - 09/05

Produce Lift-Off poster and video

Checked and provided suggestions for the poster and video

09/05 - 14/05

Learn and set up nessessary tools and configurations for developments

Installed WSL, Node.js and Typescript on my local PC
Fixed bugs related to npm on WSL 2
Learned and set up suitable configurations for Node.js runtime environment
Learned Firebase local emumerator and hosting

09/05 - 14/05

Install dependencies for back-end development

Installed Express and Firebase to prepare for development
Developed simple Express app

12/05

Read mentors' introduction slides and apply for mentor

15/05 - 19/05

Create new branch and add basic files

Created new branch for back-end development
Created basic repo stucture

20/05

Learn Firebase Admin SDK

Learned to do authentication on server side with Firebase using JWT or session cookie and manage users
Learned to design data models and do complex query with Firestore

21/05 - 23/05

Implement and test /users/* routes to handle users

Added route to get, update and delete user
Tested the behaviour of the API using Postman

23/05 - 24/05

Implement /sites/* and /pages/* to handle sites and pages

Created models to represent the relation between entities

24/05

Learn CORS

Learned about CORS and experienced with browers

24/05 - 25/05

Continue with /sites/* and pages/* routes

26/05

Learn Nest.js for back-end development in Typescript

Nest.js uses classes and decorators extensively, making the code looks similar to Spring project
Decided to stick with Express at the momment

27/05 - 28/05

Modify /users/* routes and look for packages that offer serve similar purpose as Validator in Nest.js

class-validator can be used for classes, but not for interfaces
Considered express-validator

01/06

Looking for platform to deploy the back-end

Vercel seems to be the most suitable platform for deployment
Considered to migrate the back-end to Next.js

02/06

Migrate from Express to Next.js

Migrated backend to Next.js for easer deplopment
Serverless function of Next.js works fine for the application

04/06 - 05/06

Peer review for Milestone 1

06/06

Migrate from Express to Next.js (cont)

08/06

Used `next-connect` to write API endpoints, the code looks so elegant

Note: My commit messages were terrible
next-connect is wonderful for serverless environment

09/06

Redesign repo structure, complete basic intergation

Demolished monorepo structure for Next.js

10/06

Write API specification and validators

Wrote API specifications
Started implementing validators to sanitize requests

11/06

Redesigned API endpoints

Discussed API specifications and implementation details
Changed the data models on Firestore as multiple nested collections are terrible

12/06

Implement endpoints to upload images

Tried formidable and multer to parse form data
Learned Firebase Storage
Resolved problem when parsing form data by turning off bodyParser in Next config (I was fortunate enough to find this on StackOverflow)

13/06

Get URL of uploaded images

Firebase Admin SDK does not have getDownloadUrl(), the workarounds are not reliable (more info)
Public URL was used, although it's not very secure

17/06

Add JWT authentications, transactions for CRUD and error handlers

Note: Terrible communication delayed our process ...
Authentication problem was solved trivially by attaching middlewares to verify JWT before the handling requests
Added transactions for CRUD to ensure the consistency of logic (for example: user's sites must not have duplicated names)
Wrote proper error handlers for meaningful status codes and messages

18/06

Read Jest and Firebase Emulator documentations for unit testing

Setting up Firebase Emulator was troublesome - I had to install JRE
And I kept connecting to the real databases when trying out the Emulator for unit tests. I wrote setup file (after env) to prevent this, but it did not work correctly

19/06

Start writing unit tests

Wrote scripts to generate test data
Wrote setup file (before env) to prevent the unit tests from connecting to the real databases, but it did not work correctly (again)
Wrote unit tests for users and sites. Now I know how useful dependencies injection is

20/06

Add cases for unit tests

Fixed setup files
Used beforeAll hook to setup test data, for easier modification (if my implementation changes in the future)
Added test cases to increase coverage

21/06

Add statistic for sites

Added page count for sites
Added comment count for both sites and pages
Added new test cases for sites and pages statistic
Added Firebase Emulator for unit testing
Added test cases for newly implemented features

22/06

Fix issue, and unit tests for CI

Demolished setup files, use custom testing environment (much more consistent, very nice)
Changed implementation, now allow pages to have duplicated names

23/06

Fix security problem and add CI for backend

Enforced that the decoded uid and the documents' uids must match before executing CRUD actions
I kept making typos in my action.yml files (fixed by a bunch of force-pushes)
Added even more tests to increase the coverage and used Promise.all to decrease the test time

25/06

Prepare for Milestone 2

Discussed specifications for customisation section
Wrote and reviewed README
Brainstormed for MS2 video

26/06

Optimize GET endpoints

Optimized GET endpoints, as I realized only 1 transaction is required
Looking back, I should have broken down my commits into smaller ones. I did too much in one commit
Question: Some of my commits are not shown on GitHub?

27/06

Updated project log for MS2

01/07

Improved unit tests with Firebase Emulator

Used demo project id to initialize Firebase Admin for testing
Got rid of redudant env.test.local on local machine and GitHub secrets

02/07

Implement embed endpoint

Also fixed issue #127 (get download URL for images from Firebase Storage)

03/07 - 04/07

Peer review for Milestone 2

Everyone included Machine Learning in their README :)

05/07 - 10/07

Do some minor modification to the code

Did not do anything significant during these few days

11/07

Fix auto-approve feature

Also added unit tests for this feature
codecov kept falling

14/07 - 18/07

Implement customisation feature

Some commits were repeated on repo' history?
This feature is quite easy to implement

20/07

Implement endpoint to get site statistic during the last 30 days

This features is much more difficult to implement
I tried to use a counter for this feature, but the implementation was terrible
Discovered a bug when implementing this feature - I forgot to delete subcollections

21/07

Continue with site statistic

The final implementation is not very efficient - but it works with O(n) complexity
Also fixed the bug discovered yesterday and add more unit tests to cover it

22/07

Refactor code

My code looked like spaghetti after implementing new features
Cleaned up to keep my eye sane

23/07

Minor optimization for getting site statistic

Should be at least 2 times faster

24/07

Implement the public API

For the best performance, I had to repeat code (which is not a good practice)
Fixing the code to make it less repetitive made performance decreased
I will find a way to improve it later