It would be relatively straightforward, if you had do it just once. You would sum up all the elements in that submatrix which would be of O(m*n) time complexity, where m is the number of rows and n is the number of columns in the submatrix. The question becomes a bit more complex if you have to query the sum of different submatrices multiple times. Unsurprisingly, such use-cases do come up in Computer Vision and Computer Graphics, and O(m*n) time complexity can get incredibly disturbing for repetitive calls.

Perhaps on such a disturbed day in 1984, Frank Painter rose to the challenge and wondered if he could somehow preprocess matrix A so that it would take constant time to retrieve the sum of any submatrix. The answer was yes. Not only the sum, but you can preprocess your matrix to efficiently retrieve any other statistics such as mean or standard deviation for its submatrices. When I first came across this problem, I got completely befuddled and my first thought was to calculate all of the sums for all the submatrices and store it by some index. Needless to say that would have been an astonishingly inefficient solution and Frank did not go that way. Now let’s follow Frank’s footsteps and ponder together…

In order to query the sum of a submatrix, we first need to define it. The submatrix can be defined by 4 arguments. Let’s call them x_low, x_high, y_low and y_high. If we were to draw a straight line for each of them, the submatrix would be the rectangular area constrained by these 4 lines. The number of elements in this submatrix would be (x_high - x_low + 1) * (y_high - y_low + 1) and we need to return the sum of all elements. It is clear that we have to scan the matrix, make some computations and then store the results to decrease time complexity. What if we calculate and store the submatrix sums for all submatrices defined by (0,0) and (x,y)? Compared to storing the results for each and every possible submatrix, this would be much more efficient. Let’s see if this would work.

Suppose that we can only query this interface for (x,y) which then would return the sum of the submatrix defined by (0,0) and (x,y). How can we use this method to calculate the submatrix defined by (x_low,y_low) and (x_high,y_high)? The answer becomes apparent when you draw and visualize the problem…

// returns the sum of all elements in submatrix defined by x_low = 0, y_low = 0, x_high = x & y_high = y
int sum_submatrix(int x, int y); 

int sum_submatrix(int x_low, int y_low, int x_high, int y_high) {
    return sum_submatrix(x_high, y_high) - sum_submatrix(x_high, y_low-1) - sum_submatrix(x_low-1, y_high) + sum_submatrix(x_low-1, y_low-1); 
}

Now let’s move on to calculating the submatrix sum for submatrices defined by (0,0) and (x,y). It is best to simplify the problem and visualize the solution with a 1D array. Suppose that the 1D array below is the first row of A and that we need to calculate all the submatrix sums from (0,0) to (0,4).

[1,5,1,6,8] ---> [1,6,7,13,21]

Notice the pattern in the 1D case? It all comes down to calculating the submatrix sum cumulatively. We can generalize this solution to the 2D case as below. Note that we subtract dp[i-1][j-1] to prevent it to be counted twice (which is easy to figure when you visualize your solution).

class SummedAreaTable {
  public:
    SummedAreaTable(const std::vector<std::vector<int>> & A) {
        int M = A.size(), N = A[0].size();
        dp = std::vector<std::vector<int>> (M+1, std::vector<int>(N+1,0));
        for(int i = 1; i <= M; ++i)
            for(int j = 1; j <= N; ++j)
                dp[i][j] = dp[i-1][j] + dp[i][j-1] - dp[i-1][j-1] + A[i-1][j-1];
    }
    
    int get_submatrix_sum(int x_low, int y_low, int x_high, int y_high) const;
  private:
    std::vector<std::vector<int>> dp;
}

Now let’s rewrite sum_submatrix using dp. This way we can retrieve the sum of all elements in the submatrix in constant time.

int SummedAreaTable::get_submatrix_sum(int x_low, int y_low, int x_high, int y_high) const {
    return dp[x_high+1][y_high+1] - dp[x_high+1][y_low] - dp[x_low][y_high+1] + dp[x_low][y_low];
}

Remember that this algorithm can also be used to retrieve stastics like mean and standard deviations. When you have the sum for the submatrix, it is straightforward to retrieve the mean. You only need to calculate the number of elements in the submatrix.

class SummedAreaTable {
  public:
    SummedAreaTable(const std::vector<std::vector<int>> & A) {
        int M = A.size(), N = A[0].size();
        dp = std::vector<std::vector<int>> (M+1, std::vector<int>(N+1,0));
        for(int i = 1; i <= M; ++i)
            for(int j = 1; j <= N; ++j)
                dp[i][j] = dp[i-1][j] + dp[i][j-1] - dp[i-1][j-1] + A[i-1][j-1];
    }
    
    int get_submatrix_sum(int x_low, int y_low, int x_high, int y_high) const;
    int get_num_elements(int x_low, int y_low, int x_high, int y_high) const;
    double get_submatrix_mean(int x_low, int y_low, int x_high, int y_high) const;
  private:
    std::vector<std::vector<int>> dp;
}

int SummedAreaTable::get_submatrix_sum(int x_low, int y_low, int x_high, int y_high) const {
    return dp[x_high+1][y_high+1] - dp[x_high+1][y_low] - dp[x_low][y_high+1] + dp[x_low][y_low];
}

int SummedAreaTable::get_num_elements(int x_low, int y_low, int x_high, int y_high) const {
    return (x_high - x_low + 1) * (y_high - y_low + 1);
}
    
double SummedAreaTable::get_submatrix_mean(int x_low, int y_low, int x_high, int y_high) const {
    return std::static_cast<double>(get_submatrix_sum(x_low, y_low, x_high, y_high)) / get_num_elements(x_low, y_low, x_high, y_high);
}

Lastly, you can find the much better illustrated solution here.

Next, I will look into its more advanced use-cases.

I will probably extend this blog post with a summary from all these videos. I am currently on video 9, so that will take a long time.

It was in freshmen college (2011), where I really had to learn C++ via a required class. It wasn’t through books. I learnt C++ by trial & error. Endless hours in front of the computer. Sweat & tears. It was a gradual process. I wasn’t just learning C++, but I was learning how to program. Everything, from scratch! Classes, labs, assignments & on paper midterm/final… Nowadays it seems like most schools actually teach Java or Python for programming. I was lucky. I got to use C++ throughout college and later at work.

It’s been about 10 years since I have started learning C++. I never got to learn Modern C++, as I never had to use it. I still exclusively use classic C++. That will probably change as more people/companies adopt Modern C++. Until this year (2021), I had never actually read any language-specific guidelines. All my C++ knowledge was based on my practical experience.

Books can be helpful, but I am not sure how much one can gather from them. You can’t learn a human language just from the books. It won’t workout (if you are not some sort of a savant). It doesn’t happen overnight either. Anyhow I want to give it a try - read a few books and maybe pick something along the path. I first read C++ Coding Standards for some reason, but it seems like ‘A Tour of C++’ would have been a better first book.

Here is my reading list:

1. A Tour of C++ (Second Edition)
2. C++ Coding Standards
3. Effective C++ (Third Edition)
4. Effective Modern C++
5. Large Scale C++ : Process & Architecture (Volume I)

For books, I usually delve into one topic and read everything I can find on that topic. I first make a list of books in the topic that I’d like to learn more about and batch order them. For films, I usually watch a whole series of films from one director/genre/country/era. I have accounts on both Mubi & Criterion Channel. I’d highly recommend both if you like films. I read/watch to pass/enjoy time and maybe pick up something along the path. I don’t take notes unless I really have/want to learn something.

I wanted to keep my reading/watch list on this website, but that’s just unnecessary since I already have IMDb and Goodreads accounts.

Because of my restrictive funds, I can’t buy the parts I need for my projects. Unfortunately, my GPU isn’t that good either. Sadly, machines learn very slowly without powerful GPUs. This leaves me with a few options. I can learn a new programming language to pass time or I can work on some toy projects. I even bought a freaking domain name raining.gold, which is “under construction” right now. Inspired by Stanley from the Office, I have decided to spend some of my time solving the crosswords specifically designed for software engineers. These crosswords can be found at Leetcode and Hackerrank. In other words, I decided to solve technical interview questions to pass time. I will be writing about my solutions as I want to make it a little more social - talking to yourself is a bit weird when you are at a coffee shop.

Data is produced by an event; either by an action or inaction. Devices like mobile phones produce data through user interactions over a period of time. For example, a hidden application may collect the news headlines that you are more likely to click on (it probably does). This information can be used to provide you a better “service” and recommend the news that you are more likely to be interested in. It is this “intelligence” and control over what you see that make these companies very powerful. Despite the growing public concern on data privacy, companies will continue to find innovative methods to gather intelligence. This is literally how companies like Google and Facebook make money. For them, it is an existential matter. Personally, I do not want to go back to a time where Google does not exist. I love free web search! To be honest, I am not sure if I could survive without it. It would have definitely been harder to write code. I also would rather have an independent company to provide me that service instead of the government. I mean have you read 1984?!

On the one hand, I am glad that the governments around the world are cracking down on bad data collection practices. On the other hand, I believe good data collection practices should be encouraged and government agencies should strive to make their datasets publicly available. Fortunately, the number of AI initiatives are growing. Public initiatives such as data trusts provide a platform for small and non-profit companies to be a part of the game. Given that most researchers end up working at the big tech companies, improving publicly available datasets is as equally important as funding public research in artificial intelligence. Long story short, improving publicly available data trusts can relieve the situation a bit, and all we need is competition (and love).

Accessing privacy-invasive data is another issue. AI is expected to be very useful in healthcare, especially. There are already a growing number of companies that are building wonderful applications for this reason. However, healthcare data is incredibly privacy invasive and access to data is very restrictive. We must find a way to realize the potential of AI, as it can be the “cure” to many of our problems. Instead of restricting access to privacy-sensitive data, we must strive to make them more publicly available without breaching privacy. So.. How can we train our models without sharing private information? How can we ensure anonymity?

The answer is to share intelligence rather than data. Machine learning methods like “federated learning” cultivate the shared intelligence to build powerful models in a “centralized manner”. (Unsurprisingly, research is supported by Google. Thanks, Google!) Do you like the idea? Good, then note this textbook definition down -

“Federated learning is a distributed machine learning approach that enables training on decentralized data.“

Another approach to this problem is “decentralized learning”, which is a direct descendant of anarcho-syndicalism. Fully-decentralized learning enables learning on decentralized data without a central entity that coordinates learning. In other words, it makes peer-to-peer learning possible. Therefore, improvements to this approach will provide the underlying framework to open-source intelligence.

Let’s start with the etymology of democracy. Did you know that it comes from Greek? Etymologically the word “democracy” comes from “demos” and “kratos”, “demos” as in “common people” and “kratos” as in “power”. So there is a little “power to the people” thing going on in its meaning. What, then, is meant by “democratization of machine learning” and how can we democratize machine learning?

Applying the etymology of democracy means that democratizing machine learning is making machine learning more accessible to common people. It’s not just about allowing people to interact with it, but also empowering them to build their own models. Therefore, machine learning frameworks and tools should be simple enough such that people can easily interact with them. Democratization is not just about releasing applications that use AI, but also about giving people the ability to develop their own applications.

Just 40 years ago computers were inaccessible to the majority of the “common people”. The proliferation of personal computers was truly a democratic revolution that later brought computers to our pockets. We do not look back enough to understand how far we have come in making technology available to non-technical people, and the importance of this type of convenience. Technology evolves through our interaction with it. You should not need a PhD to access it. The lower the barrier, the better and more people can adopt the technology. Of course, there should be security measures in place, but these measures should improve the accessibility of the tools. For now, you may still have to predict where machine intelligence would fail. Non-experts may fail to understand the weaknesses of the analysis, but this can be prevented with better model calibration and the expression of uncertainties. Therefore, the aim should be to provide reliable tools and technologies to people so that they can better interact with them.

Open source frameworks such as Keras, PyTorch, and Tensorflow simplify the development and training of deep learning models. In essence, these frameworks are developed collectively and in a decentralized manner. They can be pulled in any direction and adjusted for different needs. However, they still require an in depth understanding of deep learning to be useful (at least for complex tasks). It would be futile to expect non-technical people to utilize these frameworks. When it comes to tools like Google’s Cloud AutoML, the conversation is a bit different. It requires minimal machine learning expertise. A lot of people are already interacting with AI in their daily lives and these are the type of tools that will enable “common people” to produce what they consume every day. Not all companies can hire a machine learning engineer to build a model from scratch or a data scientist to analyze their data. These tools delegate the mathematically rigorous part of machine learning to the experts. It hands the power of machine intelligence to the common people and let them innovate. In the age of information, it is the only way to stay strong against bigger fish.

Here is a great answer from the author of Keras on why we should make the frameworks and tools more accessible. I would like to elaborate on the second part of his answer. In essence, his argument is that AI will automate a lot of jobs, as a result it will further increase the centralization of wealth and power. In other words, AI can potentially seize the means of production (at least in some industries) and we must democratize artificial intelligence in order to prevent inequality. Pretty simple, right?! I agree with this answer; the proliferation of open-source frameworks and tools is an important step for democratization, but it does not finalize the process.

Powerful models require mathematical rigor, adequate data and abundant computational resources. Throughout this post, I elaborated on the first requirement. It is the responsibility of the machine learning community to develop machine learning tools that are accessible, reliable and available in order to truly democratize machine learning.