Fetch Large Data in Angular in 3 optimized ways

0

This article focuses on how to optimize rendering with Angular, these techniques are actually applicable to other frameworks or simply Vanilla Javascript.

We will take a look at the following techniques:

  • Virtual Scrolling (using the Angular CDK)
  • Manual Rendering
  • Progressive Rendering

Virtual Scrolling

Virtual Scrolling is probably the most efficient way of handling large lists, with a catch. Thanks to the Angular CDK and other plugins it is very easy to implement in any component.

The concept is simple, but the implementation is not always the easiest:

  • given a container and a list of items, an item is only rendered if it’s within the visible boundaries of the container

To use the CDK’s Scrolling module, we first need to install the module:

Then, we import the module:

We can now use the components to use virtual scrolling in our components:

As you can see, this is extremely easy to use and the results are impressive. The component renders thousands and thousands of items without any problem.

If Virtual Scrolling is so good and easy to achieve, why bother exploring other techniques? This is something I’ve been wondering too — and actually there’s more than one reason as to why.

  • The way it’s going to work is very dependent on implementation: it’s hard to be able to manage all the possible scenarios with one single implementation.
    For example, my component depended on the Autocomplete field (built by the same team) and unfortunately, it didn’t work as expected. The more complex your items, the more difficult it’s going to be.
  • Another module, another big chunk of code added to your app.
  • Accessibility and Usability: the hidden items are not rendered, and therefore won’t be searchable.

Virtual Scrolling is ideal (when it works) in a number of situations:

  • an undefined and possibly enormous list of items (approximately greater than 5k, but it’s highly dependent on the complexity of each item)
  • infinite scrolling of items

Manual Rendering

One of the options I’ve tried to speed up a large list of items is manual rendering using Angular’s API rather than relying on *ngFor.

We have a simple ngFor loop template:

I’m using a benchmark inspired by js-frameworks-benchmark to calculate the rendering of 10000 simple items.

The first benchmark run was done with a simple, regular *ngFor. Here are the results: scripting took 1099ms and rendering took 1553ms, 3ms painting.

Manual rendering - Fetch Large Data in Angular in 3 optimized ways
Manual rendering – Fetch Large Data in Angular in 3 optimized ways

By using Angular’s API, we can manually render the items.

The controller’s code changes in the following way:

  • we declare our template and our container

 

  • when we build the data, we also render it using the ViewContainerRef createEmbeddedView method

 

Results show a modest improvement:

  • 734ms time spent scripting, 1443 rendering, and 2ms painting
Improvment render - Manual rendering - Fetch Large Data in Angular in 3 optimized ways
Improvment render – Manual rendering – Fetch Large Data in Angular in 3 optimized ways

In practical terms, though, it’s still super slow! The browser freezes for a few seconds when the button is clicked, delivering a poor user experience to the user.

This is how it looks like (I’m moving the mouse to simulate a loading indicator 😅):

Let’s now try Progressive Rendering combined with Manual Rendering.

Progressive Rendering

The concept of progressive rendering is simply to render a subset of items progressively and postpone the rendering of other items in the event loop. This allows the browser to smoothly and progressively render all the items.

The code below is simply:

  • we create an interval running every 10ms and render 500 items at once
  • when all items have been rendered, based on the index, we stop the interval and break the loop

Notice that the number of items rendered and the interval time is totally dependent on your circumstances. For example, if your items are very complex, rendering 500 items at once is certainly going to be very slow.

As you can see below, the stats look certainly worse:

Progressive rendering - Improvment render - Manual rendering - Fetch Large Data in Angular in 3 optimized ways
Progressive rendering – Improvement render – Manual rendering – Fetch Large Data in Angular in 3 optimized ways

What’s not worse though is the user experience. Even though the time it takes to render the list is longer than before, the user can’t tell. We’re rendering 500 items at once, and the rendering happens outside of the container boundaries.

Some issues may arise with the container changing its size or scroll position while that happens, so these issues need to be mitigated in a few cases.

Let’s see how it looks like:

Final Words

The above techniques are certainly useful in some situations and I’ve used them whenever virtual scrolling was not the best option.

With that said, for the most part, virtual scrolling using a great library like Angular’s CDK is definitely the best way to tackle large lists.

If you need any clarifications, or if you think something is unclear or wrong, do please leave a comment!

0