Bruteforced

Life of a Software Engineer at Microsoft

2022-08-06T00:00:00-07:00

Back in 2019, a few months before joining Microsoft, one of my daily activities used to be searching “a day in the life of” videos on Youtube. I was obsessed with how the Microsoft campus looked and what the NEO (new employee orientation) entailed.

Back then we didn’t know COVID existed so exploring the campus, the cafes, the office buildings, and actually meeting new engineers was all the rad.

I ended up spending almost 3 years at the company, within the same team I joined in 2019. This post is almost 2 months late now, mostly because of my supreme laziness!

If you’ve stumbled upon this post, welcome! This is a recollection of my time as a Software Engineer within the Azure DNS team at Microsoft. During this time, I went through 2 different promotions and a few “above-and-beyond” reward cycles.

What’s in it for me, the reader?

Great question! Personally, if I were you and I had a few years in the industry already, I’d skip over to the stats section to see some fun charts and observations.

That being said, if you’re new to the industry and are either starting out on a new path as a Software Engineer or have 1-2 years of experience, the retrospective and the general guidelines might be of some merit.

Without further adieu, let’s get started. As I mentioned before, I started out with Microsoft back in June 2019 and joined the DNS team under the Networking organization for Azure. Let’s chat a bit more about the team.

The team!

DNS - Domain Name System. If you wanted to explain it in CS-101 terms, you’d say it’s just a cache.

There’s not just one team that powers the DNS offering at Microsoft. There’s a whole bunch of them powering different components of the DNS stack. The team I worked on was the authoritative DNS team i.e. the machines responsible for providing an authoritative answer for each DNS query.

The tech stack 🤓

Most of the code was either in C++ or C#. This was all pretty new to me coming from a Python world (during my Master’s and my previous job which mostly involved Python and Java).

That’s pretty much the only part that the outside developer community would be familiar with as far as the tech stack is concerned. As with most companies operating at Microsoft’s scale, most of the tools and frameworks are built in-house.

Some of them get re-branded and are provided as customer offerings. However, you as the engineer get to experience these cutting-edge frameworks and tools first-hand.

Not just that, you even have the luxury of reaching out and interacting with the creators of these frameworks and services!

From monitoring services to logging frameworks to the machines running our code, everything was developed in-house.

Pros and Cons of working on a Tier-0 service

“There are only two hard things in computer science: cache invalidation and naming things.” – Phil Karlton

DNS at its essence is a cache. It is a distributed cache that maintains a key-value mapping from some name to an IP address. There is a lot going on behind the scenes, but at its core, this is what we maintain.

A DNS service is what we call a tier-0 or ring-0 service. That means we cannot have our service rely on any higher-level services like tier-1 or tier-2.

The main disadvantage of this situation is that we cannot use all the cool technology (think Azure VMs) for building, running, and maintaining a tier-0 service since a VM deployment lifecycle depends on DNS provisioning in some manner.

On the other hand, maintaining 5 9s of availability for a tier-0 service, especially one like DNS that has a humongous QPS (think millions of queries per second) is a huge responsibility!

Well, go on, tell us what did you learn in all these years?

Okay Okay! Jesus Christ, you’re impatient. As mentioned before, I ended up spending about 3 years in a single team within Microsoft. It’s not an awfully long tenure. It’s a pretty short-term stint for an engineer at a place like Microsoft.

I’ll cover my reason for looking to switch in a separate post. For now, let’s dig into things I’d probably tell my 20-year-old self (and along with that, all the young engineers out there who are just starting out).

You’re background till now doesn’t matter!

This is one of the most important things that I’ve had the “privilege” to learn in my career so far. Your schooling, education, and I’ll even go out on a limb and say, your previous work experiences don’t matter as much as you think.

There are a lot of opinions (and facts too!) about the ease of hiring from Tier-1 colleges and universities around the world. That is very true.

However, not everyone in the world can make it to that top university, college, or high school. Regardless of where you come from and what your educational background is, passion and hard work end up being the defining factor that separates a 10X engineer from an average software engineer.

I’ve had the privilege of studying under some great professors and mentors. However, if I didn’t put as much effort and heart into what I do, I’m not sure my career path would look even remotely the same.

If you love building software,
Are willing to put in the hard work to catch up and stay relevant in our industry,
And are really looking to contribute to this vast field (with a bazillion opportunities),

you’ll be just fine!

It’s tough to land an interview

The internet is full of advice on how to prepare for cracking software engineering interviews involving coding and system design elements. A lot of set patterns for preparation are out there.

But, what we don’t really talk about as much is the ability to land an interview.

What if you spend 6 months, burning the midnight oil, preparing for landing your dream job at some XYZ company, and they never even select or shortlist your resume?

Your resume is one of the most important pieces of your online profile that can help land that first email/call with a recruiter or an engineering manager. Again, a lot has been said and a lot of advice is out there on how to improve your resume so not going to add to that information here.

What I’ll mention here (or rather, really stress!) is that just applying for a job on a company’s website is not enough these days.

A company like Google or Microsoft receives hundreds of thousands of applications in a week. The competition is too tough there. Everybody is trying to make their way through the funnel and get a chance to even interview these big software giants.

What a lot of engineers don’t do, is try to build a personal connection with recruiters or engineers or engineering managers.

Building personal connections, in my experience, is much more effective to landing interviews than merely applying on job forums. If you feel your profile really fits a particular role at a company you’re interested in working at, try reaching out to some engineers or even managers in that company.

Be polite with your introductions
Share your resume
Give a TL;DR as to why you might be a good fit for a role in the company and briefly mention your industry experience.
Most importantly, thank them for the time they spent reading your message/email.

Even if one of the people you reach out to in this manner replies back, they would help short-circuit the application process and help you land that interview even before you can process happiness from their reply!

It’s ok to feel like an imposter

Preparing, interviewing, negotiating, and finalizing an offer is just the tip of the iceberg. The excitement of getting into a great company will only get you so far on the job.

My first month or so was a whirlwind of information and acronyms (Microsoft is pretty infamous for them!). Most importantly, I did feel like an imposter, a lot during the initial weeks or so.

What you need to understand is that it is perfectly ok to feel like an imposter. You simply need to redefine your expectations from the role. Instead of trying to be a rockstar on the team from day-1, you need to act like a sponge and be a very good sponge for a long time.

If you understand and accept your role as the sponge on the team, you won’t feel like an imposter. You simply need to accept the fact that every other engineer who has more tenure than yourself on the team is more experienced and probably smarter than you when it comes to doing tasks on that team.

Asking for help is not a crime

You might get mixed takes on this one. However, my personal opinion is that we should not be shying away from asking for help. When I started working in the team, my ego always dictated the amount of time I spent trying to solve a particular problem. As a young engineer, completely new to the tools and infrastructures in a company as large as Microsoft, running into issues became second nature.

After a couple of fiascos where hours or even days went by and I was stuck trying to figure out a problem only to be helped by a senior engineer in a matter or minutes, I realized that there needs to be an intricate balance between going down the rabbit hole and asking for help from others.

Doing due diligence, whether it’s for reading a piece of code or reading some documentation, is very important for an engineer’s growth and learning. However, after a point of time, we start to go down the rabbit hole and stop being productive at all. At that time, we should just reach out to senior engineers, our manager, internal slack channels (teams channels for Microsoft) and any other avenues you can find.

Communication is the key, especially to being an effective engineer in a remote-first world during the COVID era.

Chasing Impact!

Say you’ve joined a great engineering team and have started to settle in. The work you’re getting is great and is really satisfying your engineering acumen. After a year or so, things start to stagnate a bit. Not in terms of what work you’re getting to do, but in terms of your own learning experience.

At this point, you’re mostly familiar with the technology being used inside the team and so, there’s no new learning opportunity there. The team is relatively stable and is hence, not facing any scalability challenges. The new work that is coming your way is mostly along the lines of what you’ve dealt with in the past.

At this point, you might get an urge to explore other avenues since you don’t see any new work coming your way. As a young engineer, it is easy to get “bored” of the work you’re doing.

At this point, what really separates a senior engineer from others is the perseverance and the ability to think about systems from a holistic perspective. When you join a company as a new engineer, you expect to be handed new work during the initial few months. If you’re just starting out in the industry, you can expect to be handed new work 90% of the time during the first couple of years.

However, for you to really transition into a senior engineer on the team, you need to find opportunities for improvement yourself. After a year or so when you get a decent idea about the codebase and the tools and processes on the team, you can start thinking about the pain points that nobody is prioritizing on the team. Things like:

The extremely slow build times.
That broken suite of integration tests that nobody wants to look into.
Absence of good troubleshooting guides for on-call engineers.
A suboptimal design you notice for that service which has extremely high latency which is going un-noticed.
Absence of good dashboards and metrics which can help provide visibility into how the services are doing.

The list goes on and on and the more ownership you take, the more work will come your way and this is one of the sure-fire ways of becoming an SME – a subject matter expert on the team.

Everybody wants to work on that big swanky feature and build a system that maybe handles millions of QPS with 5-9s of availability and millisecond latencies. However, only a select few get to do that. In order to reach a point where you have the luxury of selecting what you want to work on, you have to work on improving the peripherals for your team.

A lot of low hanging fruits always exist within any team that are free to pick up and can be worked on the side. The best part is, they have huge scope for impact which is the one true metric that comes in handy when there are talks about promotion!

Dealing with a conflict is a key skill

The engineers and PMs on your team come from a diverse set of backgrounds and work experiences. It is only natural that you might run into conflicts and difference of opinion with them especially if you’re continuously collaborating on features with multiple engineers.

How you handle conflict is yet another invaluable skill that you must learn during your time at a company. There were multiple instances where my way of approaching a problem was completely opposite to that of another engineer(s). In these times, there are multiple ways you can go about this situation:

Go on an all out war and declare that what you’re saying is absolutely right and ignore what the other individual has to say. Don’t do this, it doesn’t work out!
Try and understand the reasoning behind the other points-of-views on the table. Even though people may present views that do not align with your thought process, always try to understand the reasoning behind the other viewpoints. Maybe there are things you’re completely ignoring and if that is the case, you can learn a thing or two in the process.
Completely succumb to what a more experienced engineer has to say considering they are …. senior? This might be the most common things we as engineers tend to do. However, every once in a while, make sure to put your point across, especially if you feel it has some merit after listening to what others on the team have to say. It not only shows good communication and listening skills, but you might end up providing a unique perspective that others might have missed.

Lastly, make sure you have fun!

This one goes without saying, have fun while you’re at it!

The grass always “seems” greener on the other side. Someone out there will always be earning more than what you are at the moment or doing better work than what you’re doing. It is very easy to get bored and have that nagging feeling to switch too soon to get better work or to make more $$.

That being said, it takes time for a new engineer to really settle down on a team.

No matter your level i.e. Software Engineer, Senior Software Engineer, or a Staff Software Engineer, it takes time to get the ground running and really make an impact in the team. Also, your morale while you’re working is more of a sine curve.

Sometimes, the projects you’re getting are really satisfying, both as an engineer and as a deliverable for the company/business unit. On the other hand, you might be doing shitty config pushes every now and then and not doing anything really productive.

The thing you have to remember is that our work is more of a marathon rather than a sprint! You need to to stick it our to really peak as an engineer within the team.

At the same time, do remember that you’re the best judge of what new work is coming your way (possibly) in the next year or so. If that is something that really throws you off and is not challenging at all, it’s time to look elsewhere :) Again, this is more of a personal recommendation rather than a well defined standard. Everyone has their own timelines for when they start to look for a switch.

Too much guidance, Sachin!

Yeah, I do realize that this post might start to feel a little bit boring at this point. This later half of the post is going to be more “stats” heavy! Don’t know what that means? Stick around to find out.

My last day at Microsoft

I had this weird idea in my mind as to what I wanted to do on my last day at the company. We tend to forget most events in life as time goes on unless we have something to show for it or recall it. You’d probably forget that amazing road trip unless you keep an album for it containing all the fun memories. Even better if you wrote a blog about it to remind you of what happens in Vegas stays in Vegas moments.

Similarly, I wanted to keep an account of sorts for myself as to what value I was able to produce as an engineer during my stint in the team. A lot of things cannot be accounted for. The things we can however take into account are your pull request data. At the end of the day, whether you’re making bug fixes or adding new documentation, it ends up being managed by some sort of version control and you do have a PR to show for it.

Now I didn’t have the time to write a tool to parse all the PRs I had merged into various codebases over my 3 years time period. So, as an extremely smart engineer, I umm, uhh.

Went ahead and made manual entries for all of them!

Yep! I added data like

Number of inserts
Number of deletes
Lines of code changed and
Number of comments
Date when the PR was merged

to a Notion database, manually.

The data was too precious to let go.

I wanted to crunch some numbers using all of this precious information I had gathered and jot down some useful information for myself and for any other engineers out there. This is by no means any sort of a 10X engineer baseline or something. This is simply the performance and outcome of my inputs over time at the company. This might look completely different for other engineers and and even if it looks completely same, they might be a whole lot more productive than myself.

Remember, these metrics by no means represent the actual contribution that an engineer made within the team. The number of lines of code changed does not translate to impact. The time spent building those design documents, leading crucial meetings with multiple stakeholders, thinking through those gnarly edge cases is also what counts as great impact :) Unfortunately, that is not really tangible.

So, here we are with some of the data I crunched from the stats I collected from my pull-requests over the years. Feel free to drop off and maybe read another fun post out there if this doesn’t seem helpful :)

Number of PRs

Looks like I worked on and merged 310 pull requests over the 3 year period.

The distribution for the number of PRs over the years looks somewhat like this:

2019 –> 29
2020 –> 129
2021 –> 120
2022 –> 32

How’d you do in 2019?

That’s the year I had just joined the team. So it naturally took some time for me to settle down and try to understand the codebase, the deployment and release processes, and a bazillion set of tools we had to use on a day-to-day.

I was really lucky to be able to work on a lot of bugs that we were facing in our production services, early on in the team. Our on-call load was pretty bad those first few months when I was on the team. That is something that turned out advantageous for me since it presented a unique opportunity to understand the code and see how and where it’s going wrong.

The ability to correlate the logs and metrics with the code was a key debugging skill that I got to learn early on and it became pretty handy as the time went by.

Something most engineers don’t realize is that focussing on your on-call shifts is one of the best ways to make an impact. You get a unique opportunity to debug that nagging issue that has been raising hell recently and if you do get to figure out the root cause, you naturally get to lead the fixes for the same. That is precisely what I did during my first few months on the team.

Most of the changes in the graphs above are focussing on adding and improving our integration test suite and patching heaps of bugs in our code base.

What about 2020?

Ah! The year we suddenly started working “fully remote”? No longer able to access all the perks and beautiful offices that the Microsoft headquarters had to offer. That was a huge mental shift for a whole lot of us not used to remote work yet.

Well, the piechart now contains changes over each quarter of the year as opposed to each month. The reason for that is my lack of knowledge in visualization charts. If I tried to display the data for each month, I ran into overlapping labels which was looking weird. Additionally, the working pattern remains interesting even if we visualize things over each quarter. The second and the third quarter seemed to be the ones where the most PRs were raised. The last quarter is usually pretty chill given all the holidays and festivals. Additionally, it feels like a slumber coming out of the holiday season and into the new year during the first quarter of the year.

3000 changes in a single PR?

March, April, and May seems to be the months where I made most of the changes for the year. There were a couple of features that I had been working on for a month or so before things got shut down. They eventually came to conclusion during March and April and had over 3k inserts.

There are multiple ways of working on big features. For these specific ones I had one-boxed all the changes in a single PR. That means all the code, configuration, and tests were included in a single pull request. The advantage of doing things this way are that it’s easy to figure out bugs in the future and revert in a single operation if needed in the future.

The biggest disadvantage however is for the reviewer. They won’t have the time and the patience to go through a PR that has 3000 or inserts. They would have to understand and spend a substantial amount of time to really review the PR. In this case, it might be the case that the reviewer either approves the pull request without doing a diligent review and this can turn out to be dangerous as some bugs might go unnoticed. On the other hand, the reviewer might not be able to get to the review for extended periods of time thus delaying the overall feature rollout.

Looking back, I’d probably break the feature down into smaller byte-sized chunks and have multiple pull-requests instead.

Slogging towards year end eh?

Not intentionally at least. The tail ender months tend to be the ones with a lot of festivals and a bunch of public holidays. So, we usually blocked out any production rollouts during this time. That being said, these were also the months where we focussed on other quality improvements in our tools and processes.

Rather than working on a big feature, we spent time working on improving things like:

Dashboards.
Build and deployment times.
Removing code smells.

Truth be told though, I usually traveled back to India to be with family during this time and that also caused a dip in the overall efficiency owing to the huge time difference in working hours.

2021, the penultimate year

This year seemed to follow a similar trend in terms of number of PRs that I worked on during the initial months of the year. March, April, and May seem to be the focus months.

This is pretty unusual if you ask me! The second quarter seems to be the one with the maximum set of PRs overall. It’s difficult to recall what went down exactly during those months since I don’t have access to the PRs themselves. But the contribution pattern of April, May, June being the best months of the year seems to be consistent across the years for me personally.

In all honesty though, I also feel that living in a place like Seattle, the months of April and May bring about the onset of Spring and you start to get slightly better weather and slightly longer days. That is also something that boosts my spirits and it would be the same for a lot of people.

An upside of working from home has turned out to be the extreme flexibility in terms of working hours. There have been times where I was able to “sneak” out early morning, do a hike and come back late in the afternoon and then work rest of the day. This might not have been possible with the requirement of going to the office.

This is however, a double edged sword. A lot of people in the tech industry, who are working from home during, tend to lose track of time while working. The sense of “working hours” is now very bleak and due to that, a lot of engineers might feel that they end up working more than they used to when working from office.

Be mindful of your work life balance, especially when working from home!

2022, the last year

I left my job at Microsoft by the end of May, 2022, just shy of that 3 year mark at the company. The process of interviewing was rigorous and looking back, I could have skipped a bunch of interviews and been more selective. However, I had a lot of fun during my interview preparation which I began somewhere around February.

Naturally, my work and performance suffered in comparison to previous years around this time.

Interestingly though, I made quite a few changes towards the fag end of my time. As a matter of fact, there were 3 set of code changes with over 2000 inserts that I ended up testing and deploying during my last week.

I had finalized an offer by the end of April and hence, I spent quite a lot of time trying to wrap up pending work items on my plate. I had a couple of big features that would get stalled in case I left them in a pending state. So, thankfully, I was able to wrap things up (no corners cut!) right before I switched.

Overall insert trends

Before wrapping up this long post, there are a couple of other interesting things that I want to document for my future self and for anyone who has been reading so far.

At the very beginning of this metrics analysis I had mentioned that I merged around 310 pull requests over the 3 year period that I worked at the company. That number might seem insignificant if I mention that these were all configuration changes or something like that (one liners essentially).

On the other hand, this number might feel pretty darn good if I mentioned that 90% of the pull requests had at least 1000 inserts/deletes.

If you’re thinking why do the deletes matter? Well, code cleanup is an important part of the overall code hygiene. The less code you have to maintain, less chances of you running into issues and easier things will be to debug.

Between 10 and 100 inserts – 122 PRs
Between 100 and 500 inserts – 71 PRs
Between 500 and 1000 inserts – 11 PRs
More than 1000 inserts – 13 PRs

This information is pretty significant because we can see that the big features i.e. the ones that tend to have a huge set of changes are few and far between. There were just 13 pull requests with more than a 1000 inserts.

Why is this information important you might ask? Well, it’s important to understand that us engineers don’t necessarily build infinitely scalable systems (or their parts) all the time. Such good and lengthy work comes with experience and also, there is limited opportunity for it. If there are multiple engineers who can work on a big ticket item, it is possible that the work might be split and you may not get to work on the “cool” stuff.

Wrapping up

That’s a whole lot of data to consume for a reader in a single post. Frankly, all of these stats might not feel to relevant or important right off the bat to an engineer. As mentioned before, this is by no means a baseline or any ballpark that you should be striving by. This is simply a documentation on my behalf for future me, so that I can analyze these trends as I move along in my career and switch across various domains.

It’s a good idea to be looking at these metrics from time to time so as to get a holistic view of your overall contributions to the team over time. Usually, us engineers work from feature to feature and never dedicate time to look back at what was done in the past 6 months or 1 year (unless we’re preparing that promo packet!).

Retrospective is a great tool for improving and paying attention to what is still missing and celebrating what has been working out well!

So, what’s next for me?

If you’ve read so far, thanks! Hopefully, there was something useful in there for you. If my future self is reading this, hopefully, you’re having a good laugh.

One of my primary aim for pursuing a Master’s degree was to get a pathway into Machine Learning. I was privileged enough to not only pursue really good courses, but also work as a Research assistant in one of the best NLP and Vision labs across the country, the Information Sciences Institute, during my Masters. Unfortunately my role at Microsoft did not involve any sort of work along the lines of Machine Learning. So, I had to put this urge to work in this field on hold.

When I started looking out for a new role, one of my primary motives was to search for opportunities that are not purely research based, but involve a good combination of Software Engineering and Machine Learning. Fortunately enough, I came across a wonderful opportunity at a company that was never even on my radar when I started searching for new roles!

As for my next gig, I bagged a unique opportunity to switch domain and enter the world of MLOps. My interviewing experience has also been quite the roller coaster ride and there would be a follow-up post documenting all of the experiences and preparation strategies that I followed.

For now, I joined Etsy’s Personalization organization as a Staff Machine Learning Engineer in the Computer Vision team. Quite the drastic jump coming from working on a Tier-0 service at an astronomical scale.

Building a Dynamic Github profile with Github Actions

2022-01-08T00:00:00-08:00

Hey y ’all, if you’ve stumbled across this post on Medium, just note that this was originally posted on Hackernoon. If you want to check out the post there, head over to this link.

A couple of years ago Github launched a neat “hidden” feature (seems like it’s still somewhat hidden these days) which allowed you to create a special kind of repository in your account.

So you see, Github allows you to create this special repository and the README for this repo, acts as the landing page for your profile. You can read more about it in this official blog post by Github.

You can share information about yourself with the community on GitHub.com by creating a profile README. GitHub shows your profile README at the top of your profile page.

A Hello-World README page 👓

It’s pretty simple to whip up a bare-bones profile page.

All you need to do is create this special repository named after your username and add a README page to it.

Make sure to make this repository public and for starters, go ahead and check the “Add a README file” box while creating the repository.

The newly created repo will contain a templated README courtesy of Github and Bob’s your uncle!

Wow, this was pretty simple! 🍦

Well, yeah! But this is like a vanilla soft-serve. It’s quick and it’s delicious.

But, that’s not what we really want now, is it?

I for one want to have “some” sundae 😝

That’s not a typo, btw (Software Enginee 😅). That’s a dynamic piece of text. Lemme go over what all is there in my Github profile and see if that is something that might interest you :)

Profile Views — Number of people who visited the profile.
Social badges — Instagram, LinkedIn, Personal Website, Medium.
Typewriter Text — That’s a very simple text/banner generator that takes up any text you want and displays it in a typewriter fashion.
Basic Information— A few bullet points about myself.
Programming Quote of the day — A daily programming quote that gets updated on the profile dynamically.
Latest Blog Posts — This article will also show up on the profile the next time you visit :)
What’s Vibin? — Displays the song I’m listening to right now on Spotify!
Hey there, Seattle! — Most recent posts from an amazing Instagram account exploring Seattle, WA.
Github Stats — There’s a whole lot to unpack here so we’ll get to this later on in the article.
The Dev card — That orange-colored card you see on the right.

The profile is not a static one but a dynamic one.

Why is that?

Because we use Github Actions — a whole lot of them — to update various parts of the README at different times of the day 🤘🏻.

If this all sounds fun to you and you want to upgrade your Github profile as well, read on!

Profile Views Counter 👁‍🗨

This one’s pretty simple to set up. All you need to do is, add the following line of code to your README wherever you want the counter to appear.

![](https://komarev.com/ghpvc/?username=your-github-username)

For more customizations to how this appears on your profile, refer to the official documentation here.

For the badges, the shields project was super handy. For ease of use, here are the relevant badge links from my profile. Feel free to change the “HANDLE” below and copy-paste the same to yours :)

We have the profile view counter and the badges set up. Woot Woot!

Typewriter Text

This one’s also pretty simple to set up. We’re not addressing the elephant in the room first!

Head over to this website and add whatever text you need along with your own customizations and finally, copy the markdown link for the same and add to your README. Easy peasy!

Basic Information

Nothing’s really fancy here. It’s just some pieces of information about myself. You don’t necessarily have to write about your technical accomplishments.

Go crazy with what you write. I added my tennis profile, my pronouns, my hiking interests, in addition to where I work.

The world is your oyster!

In general, though, try to keep it anywhere from 8–10 bullet points. Any more than that and it will start to weigh on the reader.

Basic information section from the profile.

Programming Quote of the day 📜

There are multiple ways of adding a daily quote to your profile. If you’re feeling a little adventurous, you can write a script to call an external API that fetches the most recent quotes. The API being:

[https://programming-quotes-api.herokuapp.com/Quotes?count=](https://programming-quotes-api.herokuapp.com/Quotes?count=2)2

And this is what the response looks like in the JSON format.

You can then parse this response to fetch the quotes, however many you requested — count=2 in the API — and display them in the README in some format.

In order to update the quotes regularly, you’ll need to add a Github Action that will run your script on a regular basis, maybe once each day, and update the README with the quotes.

While this might seem the programmatically pleasing way to go, the final result might require a lot more effort as the presentation is equally important for the quotes.

This seems more aesthetically pleasing than the quote itself, doesn’t it?

If you think it does, then go ahead and follow the details in this great repository.

Essentially, you need to add this image tag to your README file wherever you want the quote to appear. A new quote might appear whenever someone opens your profile page.

This is the one-liner solution that in most likeliness, does the API calls and SVG rendering in the background and simply returns an image. Notice the “quotes-github-readme.vercel.app”. Well, that’s the hosted service linked to the Github repository that is running the quote image generator server or something.

Pretty cool stuff!

Daily Dev Card 🏋

Do you like to read?

If the answer’s yes and your reading list includes some technical blogs from time to time, you might want to consider daily dot dev.

Personally, I’ve added the chrome extension to get a regular feed of articles about topics I love to read about. Not really publicizing the platform but it is a great extension and platform.

They generate an awesome dev-card for you based on your reading interests including all the badges, publications, topics you like, etc. The more you read, the better it gets.

Follow this great blog post on how to add this dev-card to your profile. Most importantly, this is a dynamic card i.e. we use Github Actions workflow to update the card at frequent intervals.

Latest Blog Posts ✏️

This one’s especially important if you’re an avid writer. There are a lot of platforms these days for writing and sharing both technical and non-technical content. There’s Medium, Dev.to, Ghost.io, and maybe your own personal website.

Basically, if you have any kind of RSS feed, you can hook it up with the README profile and keep updating it with the latest posts.

How, you might ask? Via Github Actions of course :)

This Github Action periodically replaces the following comments in your README with the hyperlinks to your recent 5 blog posts.

Follow the steps mentioned here to set this up.

Also, as a general rule of thumb, if you end up using any open source software, make sure to start it and share it! Goes a long way to help open-source developers.

What’s Vibin? 🎧 🎼

I had the most fun setting this up. Before we get to the steps, one might ask why do we need Spotify’s now playing on a Github profile page?

Well, why not? A developer is not just a bunch of commits, PRs, and open source projects.

This is just an effort to make it more personal. Go ahead and add those Strava runs, Goodreads reading lists, anything you want really.

This great repository contains all the details on how you can set this up for your README.

I ran into trouble setting this up with Vercel or Heroku based on the steps here.

The server is simply running the script “fetch_spotify.py” to get the currently playing song details for a user and then render an SVG based on a template.

Instead of a hosted service, I decided to make use of Github Actions to call the same script every 5 minutes (can’t have a higher frequency) and do the exact same thing: generate an SVG from the currently playing song.

Here are the steps I followed to set this up:

Generated the Client Id, Client Secret, and the Refresh token as mentioned in the repository linked above.
Copied the templates folder, spotify.py, and templates.json
Renamed spotify.py to fetch_spotify.py and made some changes to it. We don’t want to run a flask server. Instead, it should be invokable via the main function.
Setup a new Github Actions workflow to call the script at a set schedule

The script runs every 5 minutes and generates the “spotify.svg” file which gets placed in the repository root folder. From there, it gets picked up by the README file.

Hey there, Seattle! 🏞

This might be a bit too much on the Github profile, but, I felt it’s worth showing off the beautiful place that Seattle is which is where work and home-away-from-home is currently for me :)

For this, I wrote a Python script that uses the instaloader module to download the recent posts from a given Instagram account. We don’t need any sort of login functionality for accessing public accounts so make sure the account you choose is public.

Again, the script is attached to a Github actions workflow so that we keep the profile updated with the latest Instagram posts from the account.

You just need to set “INSTAGRAM_PUBLIC_HANDLE” as the secret for your Github Actions workflow to access and that’s about it :)

The script looks something like this:

The script downloads the recent 3 posts from the Instagram account and stores them in separate directories. Then, the README file can access them and display side-by-side.

Github Stats 🍬 🍭

This is really the meat (vegan!) of the profile. It’s the place where you really show off what your open-source contributions amount to in a crisp, easy-to-digest fashion. Mine doesn’t look like much right now but it’s a work-in-progress.

The first thing we have here is the Github stats card. This is a purview of all of your contributions so far on the platform like total stars, total PRs, commits, etc.

Follow the steps mentioned here to get the design you want for your card.

Next up, we have the top-languages card. This is something we tend to mention in our resumes. However, it’s better if Github compiles that for us based on our LOCs, right?

From the same repository as before, follow the steps to incorporate the top languages card in your profile.

Finally, there’s the big Github stats image that has a bunch of theatrical statistical stuff from your profile and contributions over the years.

That comes from a separate repository and like all of these Github stats, that is also something that is dynamically updated. The stats and top languages above come from a hosted service, but, the image/card is updated by Github Actions.

There’s a whole lot of customization around what you want and don’t want to include in the card and you can refer to the information mentioned here for that.

Personally, I got a lot of inspiration for my profile from Waren Gonzaga.

I also kinda copied the Github Actions for generating the stats card from their repository as well.

In case you wanna have a quick look at the workflow.

Other Notable mentions

That’s pretty much what all was needed to set up a great (rockstar? you tell me!) Github profile. There are a few other notable mentions that I’d like to point out though.

Carbon

Carbon is a great tool for beautifying your code snippets and adding them to your articles. The only downside is, they’re images and not scrollable like Github Gists or something.

Awesome Github Profiles

A brilliant collection for some of the greatest, inspiring Github profiles. This is where I personally started to get some inspiration for my profile.

Slackmojis

Head over here and see for yourself! Great collection of animated emojis that you can use in your profile.

Capsule Render

The footer for the profile README is this wave-like animation that keeps changing colors and gradients on its own.

If this is something that interests you, head over to the link shared before (capsule-render).

Thanks!

If you’ve made it this far, kudos to you! If you found this article useful, do make sure to share it amongst your friends and folks who are looking to upgrade their READMEs as well.

Also, here’s a link to my Github Profile repository. Feel free to fork and play around.

Into the Abyss — Crater Lake, a travelogue!

2022-01-01T00:00:00-08:00

Twenty years from now you will be more disappointed by the things that you didn’t do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor. Catch the trade winds in your sails. Explore. Dream. Discover. — Mark Twain

I had been doing local hikes around the Seattle and Redmond area for a few months now. Did some pretty amazing hikes truth be told — from the beautiful (and long!) Lake Serene to the gorgeous Lake 22. I have a thing for lakes it seems 😅.

I was out most weekends on one hike or the another but still, the heart craved for more. I wanted to do a much longer trip. Glacier National Park seemed to be a perfect candidate with a pretty good distance of 550 miles and hundreds of hiking trails to be explored.

However, the heavy presence of grizzlies and bears was enough to scare the young solo traveler in me.

I hear ya Jaimie! This one is super scary. Oh well, I guess I made the right call after all.

Coming back to the topic of this post. As I said, I wanted to go on a longer trip with a considerable driving distance to get a great road trip experience. I discovered Crater Lake back in March when I was searching for summer weekend getaways. I instantly fell in love with the place!

Note that these initial 2 sections cover a brief history of the Crater Lake and how it came to be known so and also how the Crater Lake National Park came into existence. If you are not interested in this trip way down memory lane, then you can move on to the section called “Basic Itinerary”.

A Brief History 📚

Crater Lake has long attracted the wonder and admiration of people all over the world. Its depth of 1,943 feet (592 meters) makes it the deepest lake in the United States, and the seventh deepest in the world.

Its fresh water is some of the clearest found anywhere in the world. The interaction of people with this place is traceable at least as far back as the eruption of Mount Mazama.

On August 1, 1865, several soldiers and civilians journeyed to see the legendary lake. One of the participants, Sergeant Orsen Stearns, was so awestruck by what he saw that he climbed down into the caldera and became the first non-Native American to reach the shore of Crater lake. Captain F.B. Sprague soon joined him and suggested the name Lake Majesty.

In July 1869, newspaper editor Jim Sutton and several others decided to visit lake Majesty and explore it by boat. Sutton wrote an article describing the trip for his Jacksonville newspaper. Instead of Lake Majesty, Sutton substituted the name – Crater Lake.

Crater Lake … National Park? How? 🌊

In 1870, a young man from Kansas named William Gladstone Steel unwrapped his lunch, carefully contained in a newspaper.

As he ate, he read an article about an unusual lake in Oregon. The story sparked Steel’s imagination and he vowed to see the lake for himself someday.

Two years later, Steel’s family moved to Portland, Oregon; but another thirteen years passed before Steel finally gazed upon the beauty of Crater lake.

He was so moved that he decided that it should forever be a public park.

His seventeen year quest to see Crater lake established as a national park had begun. In 1886, Steel assisted with the mapping of the lake, which had been undertaken by Clarence Dutton for the U. S. Geological Survey.

During the original survey, soundings of the lake were conducted using pipe and piano wire. The maximum depth determined by the survey was 1,996 feet (608 meters), only 53 feet off from the depth of 1,943 feet (592 meters) set by the survey of 2000.

Steel’s proposals to create a national park met with much argument from sheep herders and mining interests.

In 1893, the lake received some protection as part of the Cascade Range Forest Reserve. For Steel, this was not good enough.

He continued to work, and on May 22, 1902, Crater lake finally became a national park.

Basic Itinerary 📔 ✏️

Enough with the history of the place. The fact that this is the deepest lake in North America and also one of the clearest lakes (and bluest!) was enough to fuel my enthusiasm to visit the place.

Crater Lake is about 430 miles from Redmond. This involved roughly 6–7 hours of driving time provided there are no stops in between. I decided to make 2 stops along the way. One in Portland for lunch, which is exactly half-way through to the park and then one more in Eugene to refuel the car.

I wanted to stick to Costco for it’s cheap and good quality fuel and Eugene seemed to have the last Costco stop before entering the last leg of the journey.

The itinerary was pretty sorted except for what I wanted to do at the lake itself 😅 which we’ll get to in a moment.

So the overall plan was to start on the 14th of August early in the morning and reach the park while there’s still some sunlight left. I didn’t plan anything for this day since it was already going to be a very tiring drive. Instead, I had the entire 15th of August to explore the lake, the park, the lookout spots and whatnot. I had planned to return on the 16th. No frills itinerary!

Food 🍩 🍵

Food was definitely going to be a challenge since there aren’t many cafes or restaurants available inside the park itself and the nearest town is good 50–60 miles away which is like a couple of hours of extra driving just for food.

I didn’t fuss around too much on the food. Basically, I planned to stuff myself😂 at the Bollywood Theater — a great Indian restaurant in Portland — and then take some snacks, energy drinks with me for the one day when I’m in the National Park entirely.

There are a couple of cafes inside the park but they had huge lines are not worth it at all especially given the very minimalistic social distancing being done in there 😑

ProTip: Be sure to carry good amount of snacks and drinks, including water with you to the park. Especially given the COVID scenario, there aren’t enough places to fill up water bottles in the park, let alone any other commodity.

Washrooms 💩

Well, yeah… That… So basically, I was prepared to take a leak here and there in cases of emergency inside the park (read extreme emergency!). However, I was grateful for the good washroom hygiene that the Bollywood Theater folks had maintained which happened to be exactly half-way for me.

At the park, there are a bunch of washrooms here and there that are being actively cleaned by the park authorities given that August and September are the peak travel months for the visitors in the park. They didn’t however, have any laundry or shower options at the time.

ProTip: I didn’t need to do this while I was there but it would be great to know where these washrooms are located inside the park beforehand. Some places would be more crowded than the others. It would be great to know which locations can be more isolated and hence, slightly more hygienic.

Lodging 🏡

Its easy to sort out food and washrooms during a multi-day trip. However, it is very difficult to find good, safe lodging during these unprecedented times. You can never be too careful.

That being said, it is difficult to get a good lodging in general near the lake. There are only a couple of official park lodges and they are usually booked a year or so in advance (yes, a year!!).

Then, there is the option of AirBnbs which is a great option. The only problem is that the nearest one is about 90 miles from the lake which personally, is a pain. You’d have to drive about 2 hours just to get to the park in the day and then back again at night. Not an ideal situation for a solo traveler 😏

A great option if you’re the adventurous type is Campgrounds. The Crater Lake National Park has the famous Mazama Campground which has a whole bunch of RV spots and camping spots.

I didn’t want to camp out alone, however, I went ahead and did what seemed unconventional while planning the trip.

I booked an RV spot and decided to park my car in that spot and just sleep in my car for the two nights that I would be in the park.

Every campground site in the Mazama campground has 2 washrooms and they were very well maintained 😃. Another plus point of going with this option was that the campground was < 10 miles from the lake itself.

The beautiful sunset from the Cloucap Overlook

Sunrise, ok maybe not a sunrise 😅 from the Watchman Peak

I only started to see real crowds in the afternoon. The mornings and late evenings were almost deserted which is great for random explorations and hiking.

Day 1 — Anxiety, Thrill, Fear, Bliss! 😠 😱 👄

I was very anxious leading up to Friday the 14th. This was going to be my first multi-day solo trip which included a lot of driving. In other words, this was going to be the longest trip in terms of miles covered. And I was alone… That was a scary thought truth be told 😲

I planned to leave my place around 7a.m. initially. But then, I ended up leaving around 10a.m. since I had to take care of some bank chores that day.

It took around 3 hours for me to get to Bollywood Theater. I didn’t place an order beforehand. I thought I’ll probably park somewhere right before reaching the restaurant and place the order so that it is ready by the time I reach the place.

Anyways, I placed my order when I was 30 minutes away and was stuck in traffic crossing over to Oregon from Washington. The bridge was opening up and the traffic was holed up and I used that time perfectly to place the order. Paneer Makhani, Vada Pav, and Badam Milk. Utter bliss!

After lunch I started towards the next stop which was the Costco gas station in Eugene. Made great progress on the overall route. Next stop, Costco in Eugene which was 100 miles away.

Was so damn sleepy in between and nothing was helping. Not the water, not the songs, not the green tea, not the prospect of ramming into some car/truck. Nothing at all.

The biscuits worked however. They worked like a charm. Not sure why, but they did! Phew! It was fucking difficult to keep my eyes open.

Reached the fuel station by 6:00 p.m. and filled up the tank to the brim. Now the last leg of the journey was left. All the way to Mazama campground 😊

I started on that route and after about 15 minutes, I saw a building which I had seen from the opposite side of the road.

Hmmm, that’s weird, I thought.

Let me check the destination in Google Maps. Of course, it said Bollywood Theater!

So what happened is that the route I had saved had 3 stops. I started that route again rather than continuing the current one and so, it started to take me back to Bollywood theater which was the first stop. Goddammit!

Yeah, damn you Google Maps!

Shit just got real!

I wasted good 30–40 minutes on this because of the detour and also because after I took a U-turn, I got stuck in traffic because there was a forest fire next to the road and the traffic was badly blocked.

The campground was 170 miles away, 3 hours of drive left and I was set to reach by 9 fucking pm.

That was too late because the check-in for the campground closes at 9.

Boy oh boy did I race! I was driving at 85–90 on a 65 limit road. I was driving dangerously at times.

The most thrilling part came after an hour of drive. So I was behind 4 other cars on a single lane each side road. Wanted to overtake them but it was too risky since there were cars coming from the opposite side.

Suddenly, there was an ambulance of sorts that appeared behind my car. I maneuvered my car to the side and waited for it to go away.

Standard procedure, right? All the cars did that.

As soon as the ambulance crossed, I sped like crazy and crossed all 4 cars in a jiffy.

It was surreal, thrilling, dangerous, stupid as hell! Exhilarating to be honest.

Anyways, I was stuck for an hour right after this happened. There was an accident which pushed my eta to 10 p.m. now and I was shit scared of the night.

I kept driving and driving and driving. The sunset happened at 8:30 p.m and I had lost my mobile network around 8 I guess.

My offline Spotify playlist was a God send really. Can’t even explain but the songs in there were damn scary but calming at the same time. I was too anxious since the driving felt like I was going nowhere and just driving away into the abyss.

As it began to get super dark, my mind was flooded with so many questions.

What if the entry to the National Park is closed? (It is usually paid)

What if the entry to the campground closed? What will I do?

Will I keep driving all night long? That would be tiring, wouldn’t it?

Shall I book a spot in some other campground on the way?

“No way” I thought. Let’s keep going and see what happens.

I finally reached the entrance of the park…

And…

It was open! Phew! That was a victory, a small one, but a victory for sure.

The roads inside the national park especially were super scary. Zig-zag, no railings, 7000 feet altitude, no street lights, cars appearing out of nowhere and then there were these stupid ass boards saying:

Drive carefully. A fall here can be dangerous and can kill

Yeah, no shit Sherlock!

I finally reached the campground, found my RV spot listed on a notice board (standard procedure on their website) and quickly parked my card and dozed off for the day. I was too tired since I had spent almost 12 hours on the road. This was turning out to be one hell of a trip already!

Day 2— Lake Majestic!🚤

I got up around 6 a.m. since I wanted to see the sunrise at the lake. I noticed that morning that I was not the only one that had parked a car in the RV spot. Sure, some families were camping out but there were few folks that were also held up inside their cars.

Hmmm, not so blue 😏

In case someone wants the maps link

The 33 mile Rim Drive that travels around Crater Lake’s caldera features spectacular views of the lake and interpretive signage at all of the main vista points. I had planned my hikes and other spots along the Rim Drive for the most parts.

Watchman Peak

So the first stop on my way was the Watchman Peak. The Watchman Observation Station and Watchman Trail are located on the summit of Watchman Peak.

The observation station was designed and constructed as part of the 1920s Crater Lake Master Plan to serve as an interpretive/educational center and as a fire lookout.

The trail is just under one mile long, providing access for park visitors and rangers from the Watchman overlook parking lot (7600 feet above sea level) to the Watchman Observation Station at the summit (8013 feet above sea level).

Can you spot the observation center at the top?</a>

ProTip: This is mostly for solo travelers. Hiking alone might get boring if you’re doing it all day long with not many people on the way. Downloading your favorite songs playlists and listening to them along the hike is a great way to stay pumped and enjoy. However, I did something different this time. I listened to an audiobook of “Sapiens: A Brief History of Humankind”.

Imagine hiking in the middle of nowhere, in the middle of a raging pandemic, with no internet connection, and listening to stories of how humankind has evolved from the Neanderthals!

The Cleetwood Cove

I spent some good 2 hours just chilling in the Watchman peak area. I even dozed off in the car for sometime since I was exhausted still from the night before. The next stop on my way was the Cleetwood Cove which is probably the most packed and most popular spots along the entirety of Rim Drive.

Visitors are spellbound by Crater Lake’s blue color and mystified by its clarity. For most people, the lake’s beauty is appreciated from viewpoints around Rim Drive, but others desire a closer encounter. Some people want to fish from the shore and others plan to take a boat tour around the lake.

Cleetwood Cove Trail is the only legal access to the shore of Crater Lake. And this is the only spot where you can take a dip and swim in the lake legally!

Cleetwood Cove Trail is a steep and strenuous hike. In 1.1 miles the trail drops 700 feet in elevation through a series of long switchbacks.

As I said, this was the most packed section of the entire Rim Drive. Everybody wants to take a dip in the freezing lake on sizzling hot summer days. This day was no exception. 80% of the people were wearing masks which was a relief.

Swimming in the lake, especially on a hot summer day, can be refreshing and even a bit chilling. During summer the average surface temperature of the lake is 14 degrees Celsius.

Jam packed parking lot for this trailhead!

Hey there!, Can I sit on you?

Bluer than blue! The Crater Lake gets is one of the clearest lakes in the world which imparts the deep blue color you can see here!

Down we go ...

And here we are. This is where people usually branch off to just chill on the rocks nearby or just take a dip in the lake and enjoy the freezing waters.

Can it get any clearer?

Hello there! My faithful companion along the trip (sorry Joey!)

Mt. Scott Trailhead 🌄

The next stop along the way was the Mt. Scott Trailhead. This was the longest hike that I did that day and also the toughest. The mountain top takes you to a whooping 9000 feet which is the highest point in terms of the park elevation.

Deserves to be a bigger picture! This was clicked from a random viewpoint along the Rim Drive while enroute to the Mt. Scott Trailhead.

There, I thought I’ll help you out this time. Now can you see the observation center?

Oh hello…. random….vegetation? 😆

It was a pretty rocky trail all the way to the top. The best part about this trail was that there was nobody on it! Yes, I didn’t spot anyone while going up. There were only 5 cars in the parking lot (along the road, so not really a proper parking lot) for the trailhead.

Uhhh, where else would I go?

Almost there… Run, Run, Run.

To my surprise, there was just one more fellow traveler who was chilling at the top. That person were sitting on a rock and just enjoying the beauty of the valley and the lake. I went ahead and sat on a parallel rock, looking down upon the steep, steep valley below me.

The view was even more spectacular than what this picture can do justice to.

Cloudcap Overlook

The next stop on my route was the Cloudcap Overlook which is a simple pull-out area located on the Eastern Rim of Crater Lake.

It is the highest paved road in the state of Oregon, at an elevation of 8,065 feet. It provides the visitor with a direct, unencumbered view of Crater Lake.

I had planned to head back to the Watchman Peak to view the sunset over the lake since that is one of the best spots for sunrise and sunsets in the park.

However, the trailhead was good 30 minutes away since I was on the other side of the Rim Drive. By the time I would have reached and climbed to the top, the sunset would’ve passed.

I was disappointed since I did not have a backup viewpoint for the oncoming sunset. However, I was blown away after I view the lake from the Cloudcap overlook.

It just so happened that I had stumbled upon one of the best spots (maybe even better than the Watchman Peak) to view the sunset specifically. Again, the photo above doesn’t do justice to the orange and blue colors that we could see. It doesn’t get any better than this.

I spent a good amount of time just engulfed by the beautiful views in front of me. Note that it can get pretty chilly and windy during the evenings and a hoodie/jacket might come in handy here.

I started off back to the campground before it got pitch dark since I wanted to try and take some great photos of the roads and the pink skies.

Pinkora Porealis, anyone?

Sooo good!.

16th August — The return 😢

Well, I packed as much as I could in a single day at the lake. There are great boat tours to the famous Wizard Island as well which are not active right now due to COVID. I had a great night’s sleep after a tiring day and then I woke up again early, around 7a.m.

It was finally time to head back home

Well yeah, I wasn’t really crying but I was sad to leave this serene place behind. This was a major item checked off my travel bucket list and that left me feeling all the more confident to explore not just neighboring states and National Parks, but even places on the other side of America (Hello New York!)

Unmasking Bitmasked Dynamic Programming

2019-01-03T00:00:00-08:00

There are no coincidences in this world.

— Grand Master Oogway

Well, Master Oogway, no disrespect, coincidences do occur and I think they are just God’s way of remaining anonymous. Not just me, Albert Einstein believes this too 😛.

Source: giphy

111, that’s not really my lucky number, so to say.

However, this number filled me with joy and ecstasy when I saw it as my rank on the rankings page of the LeetCode contest number 111.

What a coincidence!

That’s the highest ranking in a weekly contest that I’ve achieved so far on the platform. The contest number and the ranking were obviously pure coincidence.

The contest usually consists of an Easy problem, 2 Medium level problems, and a Hard problem.

More often than not, the hard problem is something that requires a lot of algorithmic knowledge and prior practice to be able to pull off during the contest. This contest’s final problem was no exception to this.

Why do I say it was very hard? Have a look at the number of people who were able to solve it during the contest.

As the title of the article suggests, this problem was to be solved using Bit Masking based Dynamic Programming.

Dynamic Programming is one of the most dreaded algorithmic domains out there. It requires a lot of practice to develop intuition about a dynamic programming-based solution to a problem. I’ve always considered it to be an enhancement to a recursive solution to a problem. The main idea behind dynamic programming in layperson’s terms is:

Avoid repeated computations by caching the results.

Bitmasking as a topic in computer programming is something that I have (and I’m sure countless of other developers out there have as well) feared for a long time. This is one of those topics that will surely take you off balance in an interview and get you a rejection.

Programmers out there generally tend to avoid practicing problems related to this topic simply because it is difficult to build an intuition about it.

Optimizations related to bit manipulations occur in the most unexpected of places. With some practice, I have been able to overcome the fear, so to say, of working on bitmasking-based programming problems.

In this article, apart from describing the solution to the problem I’ve mentioned above, in detail, I will also go over some basics of bitmasking and some programming problems where it can come in handy.

As with any new thing you learn, it’s very difficult to retain theoretical concepts related to bitmasking. Retention is the best when it comes via practice. That is the main aim of this article.

0️⃣ 1️⃣ What is Bit Manipulation? 0️⃣ 1️⃣

A bit is to the computing world what an atom is to human life.

A bit is essentially the smallest unit of storage in a computer. It’s the only unit that a computer understands.

The only information that a bit can store is formed from two different states: 0️⃣ and 1️⃣. Any sort of computation that a computer performs is basically some form of bit manipulation.

Let’s look at a Wikipedia definition for bit manipulation:

Bit manipulation is the act of algorithmically manipulating bits or other pieces of data shorter than a word. Computer programming tasks that require bit manipulation include low-level device control, error detection and correction algorithms, data compression, encryption algorithms, and optimization. For most other tasks, modern programming languages allow the programmer to work directly with abstractions instead of bits that represent those abstractions.

Implementation-wise, one of the most useful and effective low-level optimizations to an algorithm is bit manipulation.

In some cases, bit manipulation can bypass looping over a data structure and give manifolds speed improvement. The only downfall to such optimizations is the code readability and maintenance.

Who wrote this piece of shitty (read lightning fast) code?

That is one scary piece of ☠code ☠

The Basics ✍️

At the heart of bit manipulation are the bit-wise operators:

And (&)
Or ( )
Not (~)
XOR (^)

These are the fundamental operators using which we can perform some complicated bit manipulation operations. Hence, it’s very important to brush up on these operators and their truth tables before moving onto some more interesting stuff.

Source: https://www.topcoder.com

The truth tables show the results for these operators when they operate on 2 bits represented by A and B. The computer has to deal with much more than just one bit of data.

The data processed by the system is generally in bytes or kilobytes or more. How do these operators work on operands represented by for e.g. 8-bits or 16-bits? In such a scenario, the operations are the same, except that they operate on each bit of the arguments. Let’s consider a simple example to clarify this.

A = 11101010  
B = 00110101

+-+-+-+-  AND  -+-+-+-+  
A & B = 00100000

+-+-+-+-  OR   -+-+-+-+  
A | B = 11111111

+-+-+-+-  NOT  -+-+-+-+  
~A    = 00010101

+-+-+-+-  XOR  -+-+-+-+  
A ^ B = 11011111

In addition to these 4 basic operators, there are two other set of bitwise operators that do come in very handy. Almost all the problems that we will look at in this article will be making use of them to give a huge boost in computational speed. These are the left shift << and the right shift >> operators.

Simply speaking, the left shift operator means multiplying a number by 2 and the right shift operator means dividing a number by 2.

Let’s look at a simple animation to show why these operators are called left shift and right shift respectively.

Left Shift Operation in Action.

For demonstrating the left shift operation, we start off with the decimal number 1 and then repeatedly multiply it with 2. As you can see in the binary representation of the resulting numbers, the only 1 in the representation keeps on shifting left one step at a time. That’s why it’s called the left shift operation.

Right Shift Operation in Action.

Along the same lines, for demonstrating the right shift operation, we start off with the decimal number 128and then repeatedly divide it by 2. As you can see in the binary representation of the resulting numbers, the only 1 in the representation keeps on shifting right one step at a time. That’s why it’s called the right shift operation.

Now that we are all versed with the basic binary operators, let’s move on to some simple use cases for these operators. We will look at a few examples below. These are not programming problems by themselves, however, they are used a lot as building blocks in a lot of algorithms.

Basic Use Cases 🛠

Counting the Number of set bits

One of the basic utilities for the operators we looked at above is to count the number of bits, set in a given binary representation.

This might not seem an important use case right now, but we will be getting into more details later on and then it will start to seem more meaningful. For now, let’s just count the number of set bits as efficiently as possible.

The first method that we will look at for this, is a bit intuitive. It makes use of the bitwise AND operator. Starting from the least significant bit, we simply check if the bit at each position is set or not and increment a counter accordingly.

The AND operator returns True iff both the bits are True. Let’s look at the code for this.

Yet another way of doing this is by ANDing the number with itself-1 until the number becomes zero. The number of steps taken to reach 0 will be the number of set bits in the original number.

The reason this works is that every time we AND the number with itself-1, one bit gets removed from the number. This goes on until the number becomes zero.

Masking and Unmasking a specific bit

Suppose we want to mask a specific bit in a binary representation_._ This simply means turning off the bit or transforming a 1 → 0 . On similar lines, unmasking simply means the reverse operation on a specific bit.

You might be wondering why this is useful at all. One of the most important use cases for masking (or unmasking) a bit is in set related operations.

We can represent a set of items as an X-bit integer, Xis the number of items in the set. Masking a bit would mean the removal of that item from the set. For a practical application of this, be patient and read on 😛.

XOR is a very versatile operator and it is perfect for the task of masking and unmasking operations.

1 ^ ? → 0  
0 ^ ? → 1

XOR outputs 1 when both the bits are opposite and a 0 when both the bits are the same.

Essentially, we can make use of the same masking variable for achieving the set / un-set operation corresponding to a particular bit (the integer in the ? is being called the masking variable).

A ^ (1 << i)

The above operation will lead to masking the bit at index i if originally that bit was set in A and the same operation will lead to unmasking the bit at index i if originally it was unset in A.

We have already seen how to detect if a bit is set or not when we looked at ways to count the number of set bits in a given binary representation.

An important thing to note here is the index. Usually, for data structures in high-level programming languages, whenever we refer to a specific index, we mean the index of a particular element in that data-structure starting from the left end.

What we are referring to an index above is from the right end (the least significant bit in the binary representation has the index 0).

That’s more than enough of the basics for now. Let’s get to some actual programming problems. This will help solidify what we’ve learned so far in the article and also help build up an intuition for solving problems using bit manipulation in general.

🔭 Missing Number 🔭

There are multiple ways of solving this problem. We can sort the given list of numbers and then iterate from 0..n and easily find the missing number. This will give us a O(NlogN) solution.

Another way of solving this problem is by making use of a dictionary. We simply add all the elements in our list to a dictionary and then we can simply search for the missing number. This is a linear time solution but it makes use of additional space.

Let’s look at how we can achieve a O(1) space, O(N) time solution like a boss 😎 by using bit manipulation.

We will make use the XOR property here to solve this problem. As mentioned before, XOR evaluates to True when the input bits are different and it evaluates to False when presented with the same bits. We are interested in the later scenario. What do you think the following evaluates to?

A ^ A

XORing a number with itself gives us 0. That’s the main idea behind our approach here.

So, what we will do is, we will XOR all the numbers in our list. Let’s call the value we get after this as A .

We will XOR all the numbers from 0..n together. Let’s call this value, B.

By doing this, all the numbers present in the original array will get XORed to their counterparts and will evaluate to 0. The only number left in the end will be the missing number.

A ^ B = missing number

☝️🏼✌️🏼 Counting Bits 🖖🏼 🖐🏼

This is one of those problems where writing down the answer for various test cases and observing the results for patterns really helps. So, we will do exactly that and use the pattern we find to directly arrive at the final algorithm.

Let’s look at the number of 1’s in the binary representation of the first 16 numbers.

0  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  
0  1  1  2  1  2  2  3  1  2   2   3   2   3   3   4   1

The pattern being formed above is the following:

An even number, E, has the same number of bits as that of E / 2
An odd number, O, has one more bit than that of O / 2

That’s all there is to see in the results above. This is the algorithm in its entirety. All we have to do is to iterate on numbers from 0..n and use the two rules above and we solved the problem like a boss 😎.

If you paid attention to the core idea of dynamic programming discussed during the beginning of the article, you’d know that this here, in essence, is a dynamic programming problem.

We make use of the result for a previous subproblem (smaller number) to calculate the answer (number of set bits) for the current subproblem.

On the face of it, we don’t need any bit manipulation as such to solve this problem. It can be solved by simple if-else clauses and a for loop.

So, this is not really a bitmasking + dynamic programming kind of problem.

Let’s look at a simple solution based on the ideas above.

This is a perfectly good way to solve this problem. For every index, i, we check the number of bits in the number i / 2 and also add 1 if the current number is odd.

We can, however, solve it in a geekier way, so to say using bit manipulation. The two operations being performed are:

division by 2 and
checking if the number is even or not.

We have already seen the use of the right shift operator, >>, for division by 2.

As for the second operation, we can simply check if the least significant is set or not. If you notice, all the odd numbers have their least significant bit set. While the even numbers don’t.

We have already seen how to check if a particular bit is set or not using the AND operator. Let’s see a geekier 🤓 version of the above code.

If you look at the runtimes for both the programs, they’re almost the same. A major chunk of the time is consumed by the construction of the output array.

Bitwise operations are always optimized and are faster than other higher level programming constructs.

⛩ Maximum Product of Word Lengths ⛩

There’s good news and bad news. The good news is that a slightly optimized version of the brute force algorithm will get your code accepted on the platform.

The brute force way is to check all the pairs of words and for each pair, check if there are any common characters between them. For all such pairs that don’t have any common characters, record the maximum value of len(word1) * len(word2).

The bad news is that this algorithm is extremely slow. Let’s look at the percentage of solutions on LeetCode which this brute force algorithm is able to beat.

Shame on you, Sachin!

That’s not the kind of stats you want to see for your submission. If you’re someone who simply wants to solve a problem, then your job here is done. Nothing more to do. But, if you’re like me and if you want to make this dog stop 😅, then read on!

Let’s look at the time complexity for this brute force algorithm before moving onto a much more optimized solution using bit manipulation.

We consider all possible pairs of words from the given array. Considering there are N words in the given array, we will get O(N²) complexity right off the bat.

Other than that, we have a dictionary containing sets of characters for each of the words in the dictionary. Considering the size of the alphabet to be 26 (lowercase alphabets only), each set can potentially be of size 26.

For each pair of words, we perform a set intersection to check whether the two words have any common characters or not. Set intersection takes linear time and hence, the overall complexity for this algorithm would come out to be O(26N²) which is essentially O(N²).

It turns out that we can’t get rid of the part where we have to consider each pair of words from the given array. So, we can’t get rid of the O(N²) part of the algorithm.

The portion that we can get rid of, however, is the part where we compare two words and see if they have any common characters. That constant 26 slows down the algorithm a lot.

An important thing to note here is that the question simply cares about common characters and not their frequency or their order.

What if we simply use a bitmask to represent the characters in a word?

What we can do here is to have a bitmask consisting of 26 bits to represents the characters belonging to a particular word. Let’s look at such a representation for a few words to make things clearer.

Bitmaps for the three words: hello, algorithm, and tweet.

Hope the above figure makes it clear what we mean by a bitmask representing a corresponding word. Once we have these bitmasks for all the words, all that remains now is to check if two words have any common characters or not.

If two words would have any common characters, then the corresponding bits for those characters would be set in the bitmasks for both the words.

Hence, all we have to do is to do a bitwise AND of the bitmasks representing two words and check if we get a 0 or not. If we do end up getting a 0, that would imply no intersection and that is precisely what we are looking for.

For e.g. 
Let's consider two words "hello" and "jack" 

The bitmask corresponding to "hello" will have the bits for 'h', 'e', 'l', and 'o' set.

The bitmask corresponding to "jack" will have the bits for 'j', 'a', 'c', and 'k' set. 

Since these words don't have any character in common, the bitwise AND of their bitmasks will give a 0. 

Had there been any common characters between them, then both their bitmasks would have set bits at the same indexes (corresponding to the common letters) and hence we would get a non-zero bitwise AND.

Let’s look at the code for this modification in the algorithm we just discussed.

Bitwise operations are super quick. Let’s look at the performance of this algorithm on the LeetCode platform to corroborate the quickness of bit manipulation.

Not bad, right? We improved the runtime for our program from 1820 ms earlier to 712 ms now. That’s a huge improvement, isn’t it? It turns out, we can improve this algorithm even further.

The improvement can be made because of the fact that two different words can have the same bitmask.

For e.g. hello and llohhel both would have the same bitmask. We can store the longest word for a given bitmask since all we care about is maximizing the product of word lengths.

Let’s look at the code after incorporating this improvement.

By doing this very optimization, the runtime comes down to 208ms 🎉🎉🎉

Set Representation using Bitmasking

We introduced an idea in the previous problem which will be crucial in the next two problems that we will discuss. Essentially, we used the idea of bitmasking to represent elements belonging to a set.

In the previous article, we considered a set of 26 alphabets and we resented that by using a bitmask containing 26 bits. A 0 at a particular index in the mask would represent set exclusion whereas a 1 would represent set inclusion.

This idea is very crucial for dynamic programming problems that deal with subproblems involving a subset of numbers. We can’t really cache a subset of numbers. A set is not a hashable data structure.

For e.g. suppose we have an array of numbers [4, 3, 6]  
Let's look at all possible subsets of this array.

[]  
[4]  
[3]  
[6]  
[4, 3]  
[4, 6]  
[3, 6]  
[4, 3, 6]

For dynamic programming problems where intermediate states are defined by these subsets, we need a memory efficient way of performing caching. 

What do we do in such a scenario?

Don’t worry, there’s a long article ahead of you!

This is where we have bitmasks come in. They are memory-efficient ways of representing subsets of elements. Let’s have a look at how we can represent the subsets above using bitmasks.

Since we have 3 elements in our given array, [4, 3, 6] we can make use a a 3-bit number for representing each of these elements. An empty subset would be represented by 000. Let's look at each of the subsets along with their bit representations.

-->> []  
-->> [6]  
-->> [3]  
-->> [3, 6]  
-->> [4]  
-->> [4, 6]  
-->> [4, 3]  
-->> [4, 3, 6]

Depending upon the problem’s constraints, we can make use of a bitmask. For e.g. in the previous problem, the alphabet set was limited to the size 26. A lot of programming problems that have small array sizes and involve dynamic programming that require you to hash subsets, usually are an indication of bitmasking-based approach.

A 26-bit mask, e.g. in the previous problem, is essentially an integer and we can simply cache that integer in a dictionary. This saves on the memory footprint of the algorithm and greatly cuts down on the time complexity for the algorithm since bit manipulation is very efficient. We can easily include and exclude elements from the subset by masking and unmasking corresponding bits from the mask.

Let’s look at another problem based on this idea before we finally discuss the star problem of this article.

🗄📂Partition Array to K Equal Sum Subsets 📂🗄

This is just one of those questions that begs you to use bitmasking. Notice the size of the array. It’s just 16 elements. If we consider a bitmask to represent the elements in an array, we’d have a 16-bit integer at our hands.

That means, to represent all the possible subsets of a given array, we’d have 2¹⁶ possible integers. We don’t really need actual subsets. All we need is a bitmask telling us the elements belonging to that subset. Based on this idea, let’s look at dynamic programming-based approach to solve the above problem.

The problem statement only asks us if a k-equal-sum partitioning is possible or not. It doesn’t ask us to return the actual partitioning. That makes the problem a whole lot simpler.

We don’t care which partition an element belongs to as long as the overall sum of a partition is what total_sum / k.

As we’ve mentioned in the above paragraphs, we will be using a bitmask to represent the elements of the array.

We will use the individual bit states of the mask to identify which numbers have already been assigned a partition and which ones still remain to be assigned one. We can only assign a number to a partition if after adding it to that partition, the total sum remains <= total_sum / k .

If the sum, S, of all the elements of the array is divisible by k, we have to complete k — 1 partitions since the last partition will automatically fall into place. In case the total sum is not divisible by k , then such an equal sum partitioning is not possible.

Why dynamic programming?

Before looking at the code for this problem based on the ideas we’ve discussed above, it’s important to understand how dynamic programming fits into the picture. We’ve already seen why bitmasking would come in handy. But where exactly does dynamic programming fit in?

Recursion is a natural fit for the problem since we have got a set of N elements and we need to partition them into k different subsets all of which have an equal sum.

Since we don’t really know what partition a number should belong to, we have to try out all the options. That means, for a given partition, we try adding all available numbers recursively (within the constraints of the partition sum) and see which choice leads us to an answer.

The recursion would be based on the following three variables:

the number of partitions remaining.
the mask representing what elements in the original array are unassigned.
and the current partition sum.

As we all know, for any problem to be classified as a dynamic programming problem, it should have the following two properties:

Optimal sub-structure ~ which simply means that the original problem must be breakable into subproblems and optimal solutions to the subproblems should be usable in a way to find the optimal solution to the main problem.
Overlapping subproblems ~ which mean there are multiple recursive calls to the same subproblems and to avoid repeated computations, we can cache the results for our subproblems.

Let’s look at the recursion tree for this problem to understand if we have any overlapping subproblems. We already have the first property satisfied.

The tuple (K, 12, 3) is the recursion state that gets repeated. K is the number of remaining partitions which in our example is 2. 12 is the current partition’s sum. 3 is the bitmask. If you consider the bitwise representation of 3, you will get 0011. That implies the two elements 5 and 7 from the set [8, 16, 5, 7] and hence the sum 12.

In the above recursion tree, we consider K = 2. That means we need 2 partitions each of sum 18. Since we never reach 18 in the tree above, the K never gets reduced. This is just a part of the recursion tree and the complete version.

We can clearly see two recursion states getting repeated. We get the same mask 0011 twice. We can simply store the result once and then re-use it later on.

Now let’s look at the code for based on bitmasking + dynamic programming.

Now we are finally ready to look at the main problem of this article. The problem that is not as straightforward in its bitmasking application as the current one and the previous one.

🔥 Find the Shortest Superstring 🔥

What do you think the simplest way of forming a superstring is?

You simply take any permutation of the given words and all of these permutations will be valid superstrings.

However, the question doesn’t ask us to form any superstring. It asks us to form the shortest superstring covering all the words.

For this problem, we will explore the idea of chaining words together. Intuitively, you can consider the idea of chaining as set union.

Let’s consider two different set of elements and then look at the combined set containing all their elements together i.e. the union set.

The repeated elements are only considered once in the union.

As we can see above, whenever we do a union over two sets, the common elements only appear once. We will adopt a similar idea with chaining two words together.

Essentially, when chaining words A and B together, we will only consider the common portion between the suffix of A and prefix of B exactly once. Let’s look at the diagrammatic representation for chaining of two words.

This is an important concept that will come in handy for solving this problem. So as to avoid repeated computations of finding out the common portion between two words which are being chained together, we will do some preprocessing.

We will find out the length of the suffix of A that overlaps with the prefix of B where A and B are being chained together. We will do this for all the pairs of words in our given list. In Python, given a string S, we can make use of S[i:] to obtain all the suffixes and S[:i] to obtain all the prefixes. We make use of this for our preprocessing below.

In the snipped above, s“words” is a list of words that we process. The function “form_edges” is called to compute the common chaining length between all the pairs of words. The function “longest_pref_suf” simply computes this common portion between a given pair of words, A and B.

Now that the idea of chaining is clear, we can move onto the next part of the problem where we explain why this problem can be solved recursively.

Why Recursion?

Now that we are familiar with the concept of chaining, we know that whenever we chain two words together, the new word formed can be shorter than the combined lengths of the two original words.

Extending this idea to all the words in the given list, we have to form a superstring that will be formed by chaining one word after the other until we are done with all the words.

The important question here is, in what order should the words be chained together?

For the three words aabc , hjaa, and chuj the chaining order hjaa → aabc → chuj = hjaabchuj is better than aabc → hjaa → chuj = aabchjaachuj since the former gives a superstring of length 9 as opposed to 12 in the latter order.

So, the chaining order determines the overall length of the formed superstring.

We will be trying out all possible arrangements for the superstrings using the given list of words and choosing the shortest one.

That seems legit, but why dynamic programming? 🤔

Instead of relying on a recursion tree for explaining the need for dynamic programming, we will look at an example that will explain the same idea.

Suppose we are given a set of 5 words [A, B, C, D, E] and we are to form the shortest superstring that contains all the words.

We already know that we are solving this problem recursively. Suppose in the middle of our recursion, we are already done deciding the chaining order of the first 3 words. Let’s say this ordering was B → A → C . Now, we need to find out the best way to chain D and E after C so that the overall superstring length is minimized.

Let’s say that in our recursion we found that C → E → D was the best chaining order. We don’t know yet if the superstring formed via B → A → C → E → D is the shortest one. However, we know that C → E → D is the best chaining order for the three words C, D, and E .

I know it’s gotten tiring, but keep reading 😛

Suppose now we arranged the first three words a little differently in our recursion (a separate recursion path) and now we have A → B → C and we have to recurse again to find out the best possible arrangement for E and D . However, we already know from above the best possible arrangement is C → E → D . Why calculate this again? This is where dynamic programming comes into the picture.

That’s it? Go on. Explain some more.

Nah, let’s now look at the the Python code bringing all these ideas together.

Uh oh! Where the heck is bitmasking in all this?

Ok ok, let’s get to bitmasking 🎉

Let’s rephrase the problem in a slightly different manner that will make the requirement for bitmasking very clear.

Given a set of N elements, we need to find a suitable arrangement for them that will minimize a certain metric. Since we are relying on dynamic programming, we will be caching results for subproblems_._

Our subproblems will be represented by the subset of these N items. As we’ve seen in this article and especially in the previous two problems, it’s difficult to cache subsets as it is.

In case the problem constraints are small, we can represent the elements of the list using a bitmask and use that for caching purposes.

After all, a bitmask is just a number.

Let’s finally get to the code for finding the shortest superstring given a set of N words.

Line 3 ~ The initial mask is composed of all 1s. That implies all the words are available for chaining. If the mask becomes 0, that means no word is left for chaining. So, the length of the superstring would be 0. This is the base case.
Line 7 ~ If you’ve paid attention to the example involving 5 words [A, B, C, D, E] from before, you know that the opportunity of caching arises when we have the two words D and E to be chained together following C.

Thus, we need to know the previous word in the superstring so far so as to attach the rest of the words. Hence,_ _prev_i__, which represents the index in the original array of the previous word in the superstring is also used for caching purposes apart from the mask.
Line 12, 15 ~ We always check all the given set of words and only consider those as options for the current step in our recursion which have not been previously used. We make use of bitmasking for this. To see if a word has been used or not, we simply check if the bit at the corresponding index in the mask is unset or not.
Line 18 ~ Makes use of the preprocessing we did earlier. For the word to be chained / attached to the word at index prev_i , we know the amount of overlap between the two. Thus, if we decide on considering the word at index i as the next word in our recursion, the amount of length it will add to our superstring would be len(A[i]) — overlap between the words A[prev_i] and A[i]

All this is fine and dandy, but wasn’t the original question asking for the superstring itself and not just the length of the shortest superstring?

Am I shying away from solving the entire problem? Of course not!

The function recurse simply returns the length of the shortest superstring and not the superstring itself. The problem statement, however, asks us to find out the shortest superstring.

If you look at the code carefully, you’ll notice a parent dictionary. At every step in our recursion, we have multiple options of words which can be chained to the current superstring (thus extending it). We use the parent dictionary to store the word at each step that gave the best answer (shortest length).

For a given mask — which tells us which words have already been chained together — and a given previous word (prev_i), the parent dictionary stores the next word that in the superstring that gives the optimal answer.

We will use this dictionary to backtrack and form the shortest superstring.

In the end, that’s one of the best sights for a competitive programmer, especially if it’s for a hard problem in a timed competition.

Conclusion 🍻🍻

Operations on bits are highly efficient and they are generally performed in parallel via optimized system level instructions.
It’s difficult, but not impossible, to write clean, understandable code involving bit manipulations.
AND , OR , and NOT are the three fundamental bitwise operators.
The left-shift << and the right-shift >> operators are used for multiplication or division by 2.
The XOR operator is a versatile operator that can be used in many different programming problems. It returns True only when both the bits are opposites and False otherwise.
Dynamic Programming-based solutions involve some sort of caching for the results of subproblems. The subproblems are represented by a set of variables. These variables are not necessarily primitive data types.
Bitmasking comes in very handy in dynamic programming problems when we have to deal with subsets and the list/array size is small. A mask (having all 0s or all 1s) can represent the elements of the set and setting or unsetting a bit can mean inclusion and exclusion from the set.
If the array/list/set size is, say, around 20, then you’d have 2²⁰ possible bitmasks which comes out to be almost a million of them. It’s not always the case you will encounter all of these bitmasks in a test case. However, that’s the maximum possible number.
We have to be careful about when to use bitmasking. We can’t have a 100-bit mask as that would be computationally intractable.
For a solution to be eligible for dynamic programming, it should satisfy the optimal substructure and the overlapping subproblems properties. Usually, through some practice, you can start to identify if the problem at hand will have a DP based solution or not.

It’s been a long article and one that I’ve absolutely loved writing. If you’ve read this far, then you’ve probably found this article really helpful. Do spread some love by sharing it as much as possible and destroy that clap button! 👏

The Perfect Programming Interview Problem

2018-11-12T00:00:00-08:00

Telephone screening interviews are a bit easier than the traditional onsite whiteboard interviews. The whiteboard interviews involve a whole lot of pressure and anxiety due to the lack of a code editor to code on. The thing that these interviews do have in common is the kind of skills they test.

Usually, a programming interview will involve one programming challenge. The candidate has to work on it for the duration of the interview. The time allotted is usually 30–35 minutes. The first 10 minutes are taken up by introductions and other things.

Given a programming problem, the interviewer usually wants the candidate to:

Give a Working Solution Come up with a working solution for the problem. This can be a brute force solution to start with. The criterion is that the candidate should be able to code up a syntactically correct program for the algorithm in that small time frame.
Ask Clarification Questions Ask questions to clarify things that were intentionally left out.
→ What’s the size of the input?
→ How many numbers can be in the array?
→ What is the alphabet size for the string given?
→ Can we use extra memory?
→ Can we modify the given input or is that read-only?
Syntactically Correct Code Once the interviewer is convinced of the solution that the candidate is describing, they are expected to write a working solution for the problem. In a whiteboard interview this solution is to be written on the whiteboard. The whiteboard obviously doesn’t have any syntax correction! That’s what makes it really hard.
Come up with better solutions If you only present a brute force solution to the interviewer to break the awkward silence, they will more often than not ask you to come up with a better solution. Unless it’s your luck day and the interviewer is convinced with the solution you proposed 😉. The kind of follow up questions they generally ask are:
→ Can you come up with a better solution? O(logn) → O(n)
→ Can you make your solution space efficient? O(1) space.
Edge Cases Even if you were able to come up with an optimal working solution for the problem, it is possible that you missed out on a few edge cases. You may have missed out on a few scenarios that don’t change the algorithm. They may affect the implementation. A candidate is expected to do extensive dry runs with the code after writing it. You are expected to try out a few test cases to find any issues they might have left in their code.
Complexity Analysis If there is still time remaining in your interview and the interviewer seems to be satisfied with the code you came up with, they might ask you about the time and space complexity of your solution. Hence, complexity analysis is also a critical skill set required to crack these programming interviews.

Yes, it is indeed overwhelming.

This post is not really about tips and tricks about preparing for and attempting a programming interview. There are a lot of good posts out there for this.

I recently came across a programming problem that I had to solve within a time span of 1 hour. It was a part of a programming contest being held on Leetcode. I found this problem to be a great candidate to be asked in a programming interview.

I will go through the problem in detail here and discuss my reasons for why this is a good interview problem. I will do my best to try and relate it to the points mentioned before.

🐍 Let’s Play Snakes and Ladders

This question was a part of a recent weekly programming contest held on LeetCode. It’s a 1.5 hour contest with 4 programming challenges of varying difficulty.

This problem was marked as a medium difficulty problem. Being able to solve it during the timeframe is a big thing as you will realize after going through the article.

It’s a big question and I would definitely urge you to go through it before reading on.

The part of the problem that I want you to focus on is the following:

Return the least number of moves required to reach square N*N. If it is not possible, return -1.

Looks like a typical Dynamic Programming problem

Or is it?

If you have been practicing dynamic programming problems for a while, it should be a no-brainer that problem statements similar to the one above usually employ the dynamic programming paradigm.

The reason we say this is because for a problem to be solvable using dynamic programming, it should have certain characteristics.

The problem should be breakable into smaller subproblems. The subproblems can be used to solve the main problem. Optimal solutions to subproblems should help us find the optimal solution to the main problem. This means a problem should be solvable recursively.
The second most important property is that of overlapping subproblems. Essentially, what dynamic programming does for us is that it helps us reuse optimal solutions for subproblems. In case we have multiple recursion paths with overlapping subproblems, we should only calculate answers for them once. Then onwards reuse them. This is the caching part of dynamic programming.

The reason why this problem fits the bill for dynamic programming is because of the following components of the problem:

We have a grid where each cell has a specific number. That number can help us define the state of a dynamic programming solution.
At every cell in the grid, we have 6 options available. These represent the 6 dice values that we can possibly get on playing snakes and ladders. Naturally, these 6 steps help us transition from one state to another state. This represents the “breakable into subproblems” part of the dynamic programming requirements.
Since we can break the problem down into smaller subproblems, we already have one requirement satisfied. If you think carefully, it is possible to reach the same cell on the grid multiple times via different routes. Let us look at two possible ways of reaching the cell value 22 starting from 1 .

Route Followed: 1 → 4 → 10 (Snake Up!) → 22

The path followed by the player in the above GIF is as follows:

--> Start at the node marked 1  
--> Dice value 3, hence move to node marked 4  
--> Dice value 6, hence move to node marked 10  
--> Since there was a snake at node 10, move to its head and hence the node valued **22**.

Route Followed: 1 → 4 → 9 → 14 → 17 → 22

The path followed by the player in the above GIF is as follows:

--> Start at the node marked 1  
--> Dice value 3, hence move to node marked 4  
--> Dice value 5, hence move to node marked 9  
--> Dice value 5, hence move to node marked 14  
--> Dice value 3, hence move to node marked 7  
--> Dice value 5, hence move to node marked **22**

As we can see from the two paths above in a sample snakes and ladders grid, there are two ways of reaching the node valued 22 . There are a lot of other ways as well. To put across the point of overlapping subproblems, the two shown here are enough.

Now that we know a single cell defines the state of our dynamic programming solution and there are multiple ways of reaching the same cell, this implies that our second requirement for a dynamic programming problem is also satisfied i.e. overlapping subproblems. Once having calculated the answer for a given subproblem, it shouldn’t be computed again. It should just be re-used.

Let us now look at a formal dynamic programming formulation for this problem.

dp[i] = Minimum number of steps from cell(i) to reach the destination cell.
dp[i] = min(dp[i + 1], dp[i + 2], ... dp[1 + 6])  
We have to choose the move that gives the minimum number of steps.

The above formulation looks pretty clean and we can proceed with it.

However, there’s a major flaw in the above formulation. 😉

The way we have modeled our problem here is something that will continue till the end of the article. So, let us first define the model of our problem. Then continue with describing the flaw in the dynamic programming approach.

Graph Model

We can consider each of the cells in our snakes and ladders grid as a node in a graph. The six possible moves that a player can make from a given cell represent the edges. These edges are directed edges. A move that takes us from cell i to cell j , doesn’t necessarily take us back from cell j to cell i . So, for the problem formulation we have:

A Graph G(V, E)
Each cell in our snakes and ladders grid represents a node in the graph. Naturally, there are N² nodes in the graph.
Every move from cell i to cell j represents a directed edge in the graph from node i to node j .
Since, for every cell in the grid, we have at most 6 moves, this means the total number of edges in our graph would be 6N².

Let’s consider a small grid and its corresponding graph for more clarity.

And the corresponding graph for this grid would be.

Edges for nodes 1 and 2 in the graph. As you can imagine, the graph becomes complicated very quickly because of the number of edges. There should be 4 * 4 * 6 = 96 edges in total in the graph since we have 16 nodes and each of them will have 6 edges in total. But, the final nodes 11–16 will have fewer edges. e.g. 11 will have just 5 edges. Similarly, 16 will have 0 edges. Hence, total edges will be 96 — (1 + 2 + 3 + 4 + 5 + 6) = 75.

The Flaw in our DP formulation

Now that we have defined our graphical model for the problem, we can look at the flaw in our dynamic programming formulation. The formulation we looked at was the following:

dp[i] = Minimum number of steps from cell(i) to reach the destination cell.
dp[i] = min(dp[i + 1], dp[i + 2], ... dp[1 + 6])  
We have to choose the move which gives the minimum number of steps.

If there were no snakes involved in the problem, then the above formulation would have been complete in itself. The problem induced by the snakes is that of loops in our graph. A snake can bring us back to an already visited state in our graph.

Loop introduced by snakes in the graph representation.

The problem this creates in our formulation is we can’t really consider a single cell in the grid to define the state of our dynamic programming formulation.

Let’s see why that is the case via an example and then we will see how to fix this problem by a different formulation.

The player moves from cell 1 to 22 and follows the path 1 → 5 → 16 (Snake Up!) → 22.

This is one of the possible routes of reaching the cell 36 starting from the initial cell 1.

Suppose that we want to find out the shortest number of steps to take from cell 22 to reach the destination cell 36 . In the above case, the player went from 1 --> 5 --> 16 (Snake Up!) --> 22 . As we can see in the above figure, the player will come down to the cell 11 due to the snake. Then take one step ahead i.e. to cell 12. Finally, take the snake up to the destination cell 36 . This is just 1 step.

Following the snake from 22 --> 11 and from 12 --> 36 is not really a move. The actual move is from 11 --> 12 . Hence, the minimum number of steps required to move from 22 to 36 is 1.

Now consider the following state of the grid when the user had reached 22 .

The player moves from cell 1 to 22 and follows the path 1 → 6→ 11 → 16 → 22.

The above scenario is also possible during recursion. In this case, when the player reaches the cell 22, they don’t have 6 options in front of them. The reason for that is, the cell 22 is the starting point for a snake. If a cell is a starting point for a snake, then it has to be followed to its head. The player will end up landing in cell 11 which has already been visited. This is a loop in our recursion. Since we cannot make any other move from cell 22 , there is no way of finding out the minimum number of steps from 22 to 36 .

The problem is:

A cell alone cannot represent the state of our dynamic programming formulation. We also need to keep track of the cells visited prior to the one at hand. A combination of these two would define a unique state in our DP formulation.

Updated DP formulation

The updated DP formulation, as mentioned before will have to take into account the visited cells and the current cell where the player is at.

Since, the state of a dynamic programming problem is generally used as a key to a cache that stores the results for various states (memoization), simply keeping a dictionary or a set of visited cell nodes is not feasible. These data structures are not hashable.

An alternative approach which can be adopted is bit-masking. We can make use of a bit-mask to mark the cells of the grid that a user has already visited before visiting a particular cell.

Let’s consider if this is even a feasible approach to follow. So, for a grid size of 20-by-20 , there would be 400 cells and we would need a bit-mask of 400 bits. Each of these bit values would represent if that particular cell is already visited or not during recursion. Since each of the bits can have two different values: 0 and 1 , there are 2^400 possible grid states. Multiplying it with 400 since we also need the current cell, we get a whopping (2^400 * 400).

Such a huge number of possible states will not work out and is not tractable at all. This is the reason that we have to move on from a dynamic programming solution to something else because of the sheer number of states in the problem formulation.

Breadth First Search to the Rescue! ⛑

Let’s try and formulate the problem in a slightly different manner. We already said that the cells in our grid represent the nodes in a graph. The 6 possible moves represent the directed edges to other nodes in the graph.

We want to find the minimum number of moves of reaching the destination starting from the initial point on the grid. This boils down to finding the shortest path in an unweighted graph.

You can either say this graph is unweighted, or you can say that all the edges are of equal weights. Hence the weights can be ignored. Finding the shortest path in an unweighted graph is a pretty standard problem. The most standard algorithm for solving it is the breadth first search algorithm.

We don’t need any special state in our graph for breadth first search like we needed in the case of dynamic programming. We can definitely have multiple ways of reaching a particular node from the starting position. However, the first route that is discovered in the breadth first search algorithm is the shortest one. That is the basis for the algorithm.

That means the first time we encounter a node/cell during our search, the number of moves performed till then would be the minimum number of moves required to get to that state/cell/node from the starting position.

After that if we encounter the same node again, we can simply ignore it. We would already have the shortest path to that node by then. This ensures that we don’t process any node of the graph more than once in breadth first search.

The breadth first search algorithm makes use of the queue data structure. The queue contains the nodes of the graph at a particular level at any point in time. Since we will only process each node exactly once, the maximum possible size of the queue can be O(N). That’s the upper bound on the size of the queue. This approach to the problem is very tractable and is in fact the optimal way of doing it.

Let’s look at the pseudo-code for the algorithm that we just proposed for this problem. Then we will look at the implementation for the same.

1. Initialize a queue for the BFS algorithm. Let's call it "Q".  
2. Add the first cell to the Q. Note that we also need to keep track of the level of nodes in our graph. The level will tell us the minimum number of moves made to reach a specific node. The level for the initial node would be 0.  
3. Process until the Q becomes empty.  
    a. Remove the front element of the Q. Let's call it "node".  
    b. For each of the 6 possible moves from "node", add the ones that have not been processed before, to the Q.  
4. If we encounter the destination node during the processing, simply return the level value at that point.

There are a lot of interesting points that we should address before looking at the implementation. These are from my own attempt at the problem. Some of these might seem too simple to have been mentioned. I wanted to put across all these cases. They are important to they way you write the code for this algorithm we discussed.

Cell Value to Row and Column Mapping 👺

The first of these points that is important to think about is the mapping from the cell values to the actual row and column numbers.

Remember, we are given a grid of some values and each cell of the grid has a numbering. The numbering system is written boustrophedonically from bottom starting from the bottom left of the board, and alternating direction each row.

The way the implementation has been done here initially is by considering the actual row and column numbers and then somehow using them to progress in the grid.

According to the problem, if a cell contains a snake, then the value in that cell is the destination cell where the player would land. We need to map the destination cell value to the corresponding row and column number.

The movement for the row is always easy. We either move in different columns in the same row or we can shift one row up. That’s it for the row.

As far as the column is concerned, there are two possible directions of movement. The question is especially tricky. The numbering of the cells alternates from one row to another. The movement within a row (for the various moves) will also have alternating directions in alternating rows e.g. for a 6-by-6 grid, the movement will be to the right for the bottom row and it will be to the left in the second last row.

For a 6-by-6 matrix, the row number 5 (considering a 0-based indexing of the matrix rows and columns) would be the one containing cells from 1 .. 6 and the row number 4 would contain the cells from 7 .. 12 .

This means, for the even numbered rows, the direction is to the left. For the odd numbered rows, the direction is to the right. However, This mapping gets reversed when N is odd. Consider the scenario in a 5-by-5 matrix.

The last row in this matrix is 4 and that row has a direction to the right. This means, in these cases, the even numbered row has a direction to the right. The odd numbered row has a direction to the left.

Implementation wise, we can use something like the following.

even_direction = 1 if N % 2 != 0 else -1  
direction = even_direction * (1 if row % 2 == 0 else -1)

In our implementation, we consider a value of 1 for a direction to the right and a value of -1 for representing a direction to the left. The reason for this is the simplicity it induces in simulating movement. We can simply do something like:

for move in range(6):  
     new_cell = cell[r][c + direction]

The c + direction would either go towards right or left. This is because the value of direction will either be 1 or -1 depending upon the value of N .

Now we can look at the mapping of a cell value to the corresponding row and column indices.

Note that the following code is only to be executed when we find a snake in one of the cells. board[row][col] will contain a -1 to represent a normal cell. The following code is executed when a snake is found i.e. board[row][col] != -1

(1) value = board[row][col]  
(2) snake_dest_row = N - (value / N) - 1  
(3) new_direction = even_direction * (1 if snake_dest_row % 2 == 0 else -1)  
(4) snake_dest_column = (value % N)  
                          
(5) if new_direction < 0:  
(6)    snake_dest_column** = N - 1 - snake_dest_column  
                          
(7) next_cell = (snake_dest_row, snake_dest_column)

Line 1

Gives us the value stored at the current cell. The current cell is represented by row and col .

Line 2

Using the value stored in the cell we find the row where that cell would be located. Remember, the value represents the cell we need to move the player to since they encountered a snake.

Line 3

Since we figured out the row where the new cell (destination cell) would be located, we can also figure out the direction of cells in that particular row. We discussed directions to the right and left depending upon the value of N, above.

Line 4, 5 and 6

Represents the offset for finding out the column value. For a row which has a direction going to the right, the offset itself will represent the column index. If the direction is to the left, then the offset will be from the right. We find the column index using N — snake_dest_column — 1 .

Problematic Cell Values 😠

As mentioned before, the cells either contain -1 or they contain a snake. The way a snake is represented is by a cell value that represents the destination cell that a player will reach by following that snake.

So, while handling the snake case, we need to be able to get the row and column number where the corresponding cell will be located.

We saw in the previous section the way we can do that. There’s a small mistake in the code, however.

Consider the scenario where we have a 4-by-4 grid. A particular cell contains a snake that takes the player to (or whose destination cell is) 8 . Let’s see the corresponding row and column values that we get by using the code above.

value = 8  
even_direction = -1  
snake_dest_row = 4 - (8 / 4) - 1 = 1  
new_direction = -1 * (-1)  = 1
snake_dest_column = 8 % 4 = 0
next_cell = (1, 0)

Clearly, this is not the correct cell. The correct cell representing the value 8 is (2, 0) and not (1, 0) . The way we fix this is by not considering the value as it is but by subtracting 1 from it. Then this problem doesn’t arise.

value = board[row][col] - 1

Making one move per Snake 🐍

One of the last important components of the implementation is adhering to the rule mentioned in the problem statement which states that “you only take a snake or ladder at most once per move”. If the destination to a snake or ladder is the start of another snake or ladder, you do not continue moving.

The way BFS works is, we process the node currently popped from the queue by looking at its adjacency list. Then consider all the nodes that have not been processed yet and then add them to the queue.

So, for a given cell, there would be 6 adjacent nodes. Except for some cases where it’s not possible to have 6 moves in all. We have to consider all them and add the ones not processed yet to the queue.

No special handling is required for the moves that land up in cells that don’t contain a snake. So, all we do in this case is:

if board[row][col + direction] == -1:  
    process it normally

In case the move we make lands us with a snake, instead of considering / processing the node where we are after the move, we consider the snake’s destination cell and add that to the queue instead if it was not processed before. Let’s consider an example for this.

The 6 moves when the player is at the cell 1

As you can see in the figure above, we don’t consider the node 4 when processing the 6 moves corresponding to the cell 1 . Instead we consider the eventual destination for 4 i.e. 11 .

If suppose there were a snake from 11 to some other node, then we will not process that. We already processed one snake move from 4 to 11 . There cannot be a continuation of snake moves here. That is an important thing to note.

Let’s look at the implementation that brings all these ideas together.

Can we get rid of the “direction” thingy? 😬

It turns out there’s a simpler way of doing what is accomplished using the even_direction variable above.

Essentially we want to know, given a row number, what is the direction of cells in it. Is it towards the right or towards the left?

The reason we need this information is because that helps us decide the final column number when we encounter a snake.

Instead of relying on the even_direction variable, we can use a simple check like the following:

if N % 2 != row % 2:  
   direction = right  
else  
   direction = left

This covers all of the cases for us. Remember, the direction for a row depends upon the value of N.

N is even and row is even, then N % 2 == row % 2 and hence the direction will be towards left.
N is even and row is odd, then N % 2 != row % 2 and hence the direction will be towards right.
N is odd and row is odd, then N % 2 == row % 2 and hence the direction will be towards left.
N is odd and row is even, then N % 2 != row % 2 and hence the direction will be towards right.

Mapping the other way around

In the above implementation, we worked with row and column numbers. Then mapped a cell value to a row and column number whenever required (in case of snake that is). We can also solve this problem by working the other way around.

It’s much simpler to work with cell numbers instead. Starting from 1 and for every cell we have to consider the next 6 cells.

All we would have to do is then map those cell values to the corresponding row and column numbers (again, to check the presence or absence of any snakes). We already saw how to do that in the above implementation as well.

This shortens the code. The solution I explained above is the one I came up with when thinking about the problem. I wanted to explain it as it is to make things clearer. This might not be the best way to go about writing the code for the algorithm describer. It helps describe all the caveats.

Why again this is the best programming interview problem?

You might be lured towards a dynamic programming based solution (like I was 😛). In that case you need to reason and get out of that loophole. DP doesn’t work here.
Coming up with a BFS based approach here is super critical. A good grasp of your graph based algorithms is necessary. The important thing is mapping the problem to finding shortest paths in unweighted graph.
Writing the code for mapping cell values to corresponding row and column numbers can be super confusing especially under time constraints. It’s not a big piece of code but you need to be able to think it through properly.
Handling the edge case where we get incorrect mapping if we consider cell values as it is. We had to subtract 1 from the cell value before mapping it to the corresponding row and column. Failure to do this will give incorrect results.
Writing correct, working code during time constraints is also a tough-ish task. You really need to keep your cool.

All in all, this problem tests a lot of different aspects of programming and thinking ability of a programmer. I feel that is one of the most important aspects to test during such interviews.

Not that these interviews are the best way to hire the candidates. Since they exist, this problem brings out a lot of different qualities that interviewers look for in candidates.

If you found the article to be useful, do share it as much as possible 🚀

Fun with array rotations

2018-09-17T00:00:00-07:00

Arrays are one of the most versatile data structures out there. Arrays form the basis of so many applications and numerous algorithms and data structures are based on them.

For e.g. the binary search algorithm works the way it does because the array data structure provides us with random access of the contents. If we take away the random access, we can’t execute the famous binary search algorithm with the same time complexities as before.

Similarly, we have the priority queue data structure which is again based on the array representation of a complete binary tree and the heap in itself is a root of a so many applications out there.

This particular article will deal with programming problems related to rotation in arrays and strings. This is a fairly interesting domain of problems and we will look at a bunch of different problems based on the concept of rotation in the arrays.

Before starting off with the actual problems, let us first look at some examples of rotations and try and answer the following questions about rotations:

What is a rotation ?
How many rotations are possible for an array containing N elements ?
What is the time complexity of rotating an array by one element ?
Some Python magic to implement rotations.

What is a rotation?

The diagram below will make it fairly clear as to what rotation is actually.

Essentially, we remove the first element of the array and we place it in the end and we shift all of the remaining elements one step to the left. This is an example of left rotation.

Similarly we can have right rotation.

Number of Rotations?

The diagrams above make it pretty clear. Whether we have left rotation or right rotation, for an array of N elements, we will have N possible rotated arrays (including self).

Time Complexity of Rotation?

Essentially what we do when we rotate an array is we remove the first element (considering we are talking about left rotation) and we shift all of the remaining elements one place to the left and finally we insert the element we removed from the first location at the very end of the array.

Since every time we have to do a rotation step, be it left or right rotation, the remaining N-1 elements have to be shifted as well to accommodate the rotation, the time complexity of this operation is O(N).

Python Magic! 🧙‍

There are many ways we can go about this. We will only showcase methods for doing left rotation and the right rotation can be achieved in similar ways. So that is left as an exercise for the reader.

This is the most basic way of implementing one step of left rotation on a given array. We simply place the first element in the very end and before we do that we shift each of the remaining elements i.e. starting index 1 (for a 0 based indexing of the array), one step to the left.

This approach actually ends up modifying the underlying array. A lot of times we are only interested in the rotated version of the array or we are interested in all of the rotations of the given array, however, we don’t really want to modify the underlying array. You can say that the given array is a read only data structure.

If you notice carefully, in order to do left rotation for the Nth time, you would need the result of the previous rotation. So, for e.g. if the original array given to us was [1,2,3,4,5] and you follow the method listed above, after one rotation this would become [2,3,4,5,1] and then we can perform one more left rotation on this and get [3,4,5,1,2] .

By following the above method, it’s really difficult to obtain the array that remains after N left rotations.

Let’s look at an interesting way using which we can achieve this.

The trick here is the modulo operation. If you notice the rotated arrays, its like the starting point for the rotated array is actually some index i in the original array. This index i can be determined by the number N which represents the number of rotations we want to perform on the given array and then return the result.

Also, as you can imagine, N can be large as well. It can be larger than the length of the original array. However, after a certain point of time, the rotated array start to repeat itself. So, for an array of size N, after N-1 rotations, the next rotated array we get is the original one.

To understand why the modulo operation here works, have a look at the diagram below which shows a few rotations.

Hope this diagram gives you enough clarity as to why we can simply do the modulo operation and we can directly get the array after N rotations have been performed on it.

Instead of writing the code like it has been shown in the code snippet earlier, we can also have a one liner for this in Python.

Now that we have a sense of rotations and we know how to play around with our array, we can finally look at some interesting problems centered around the concept of rotating an array.

Rotate Strings

Suppose you are given two strings A and B, which may or may not be of equal lengths 😛 (did you miss this ?), and we are to return true if any specific rotation of the string A can give us the string B.

In the diagram below we consider two strings A = abcde and B = cdeab and after two rotations the string A becomes equal to the string B. So in this case we return True

A simple check that will definitely return False is if the lengths of the two strings are different. In this case no matter what rotations we do, the strings can never be equal.

A very naive way of solving this problem is to find out all the rotations and then do string matching with the string B to see if the two strings become equal. We’ll look at the solution first, then we’ll see it’s complexity analysis and finally we will look at how well it fares among other solutions on the leetcode platform.

Time Complexity: O(N²) because for every rotation we do a string matching of two strings of length N which takes O(N) and we have O(N) rotations in all.

Space Complexity: O(N) because we create a new list per rotation.

On the leetcode platform this solution performs poorly as expected.

It turns out that we can do better than this.

The idea here is to append the string A to itself and then check if the string B is a substring of this extended string A + A

Why do we do this you might ask?

Well, it turns out that if we append a given array / string to itself, the resultant array or string covers all of the rotations of the original array. Let’s have a look at the diagram below to understand how this concatenation operation effectively yields all possible rotations. The string we will consider for this diagram below is abcde and so after concatenating this string with itself we get abcdeabcde

Figure showing all possible rotations for string “abcde” covered by “abcdeabcde”

Now if the string A or any rotation of A does in fact equal the string B, then the string B would be a substring of this enlarged string 2A.

Time Complexity: O(N) because all we are doing is string matching between a string of size N and another one which is 2N.

Space Complexity: O(N) because we have to create a new string of size 2N to accommodate this enlarged version of the string A.

This algorithm is much faster than the previous one and much shorter to implement as well. It’s a one liner in Python 🙈.

I’d say that’s fast enough!

Let’s move on to another interesting problem that seems simple enough but has a bunch of caveats to consider before we get the perfect solution.

Minimum in Rotated Sorted Array

Let’s take a look at another interesting problem from leetcode.

So the question simply asks us to find an element in an array that is

sorted,
rotated and apparently
doesn’t contain any duplicate elements.

A very brute way of solving this question is to search the entire array and find the minimum element. This approach would simply ignore the fact that the given array is sorted and this is the naive approach to solve this problem. So first let us look at a simple linear search based solution for this problem.

Time Complexity: O(N) if there are N elements in the given array.

Space Complexity: O(1)

This is actually interesting. An O(N) solution gives us the best execution time on leetcode. However, it turns out that we can do way better than this as far as the asymptotic complexity is concerned.

The fact that the given array is sorted is a huge hint in itself. Since the array is sorted and we are to find an element in the array, we can use the binary search paradigm.

However, the array is rotated. So simply applying the binary search won’t work here.

In this question we would essentially apply a modified version of binary search where the condition that decides the search direction would be different than in a standard binary search.

In a standard binary search algorithm we do the following

while left <= right  
    mid = (left + right) / 2  
    if element == middle element:  
        return mid   
    elif element < middle element:  
        move to the left i.e. [left, mid - 1]  
    else:  
        move to the right i.e. [mid + 1, right].

Since the given array is sorted, we can definitely apply the binary search algorithm to search for the element. The only thing is, that the elements have been rotated and that is something we have to account for.

How do we check if the array is even rotated or not in the first place?

If the array is not rotated and the array is sorted in ascending order, then

last_element > first_element

In the above example 7 > 2. This means that the array does not have any rotation. In this case we can simply return the first element of the array as that would be the minimum element.

However, if the array is in fact rotated, then there would be a heartbeat formation happening somewhere in the array. Let’s look at what we mean by a heartbeat formation.

If you look at the elements of the array above, they are in increasing order as expected (because the array is sorted in ascending order). However, after the element 7, there’s a sudden drop and then the values start to increase again. This is the heartbeat structure we are talking about.

In the array given above 3 < 4. Hence the array is rotated. This happens because the array was initially [2, 3 ,4 ,5 ,6 ,7]. But after the rotation the smaller elements[2,3] go at the back. i.e. [4, 5, 6, 7, 2, 3]. Because of this the first element [4] in the rotated array becomes greater than the last element.

The heartbeat structure that is evident from the question means there is a point in the array at which you would notice a change. This is the point which would help us in this question. We call this the Inflection Point.

An important property of the inflection point that would be critical in solving this question is:

All the elements to the left of inflection point > first element of the array.
All the elements to the right of inflection point < first element of the array.

Let us now look at the algorithm to solve this question before looking at the implementation.

Find the mid element of the array.
If mid element > first element of array this means that we need to look for the inflection point on the right of mid.
If mid element < first element of array this that we need to look for the inflection point on the left of mid.

We stop our search when we find the inflection point, when either of the two conditions is satisfied:

→ nums[mid] > nums[mid + 1] Hence, mid+1 is the smallest.

→ nums[mid - 1] > nums[mid] Hence, mid is the smallest.

Time Complexity: O(logN) because all we are doing here is relying on our good friend, binary search and thus making use of the sorted nature of the original array.

Space Complexity: O(1)

Can’t do better than that now, can we ? 😉

The big catch in this problem is that there are no duplicate elements in the array. What if there are duplicate elements in the array ? Can we still follow a similar approach to solve the problem?

The answer to this question is yes and no. The same concepts that we discussed above apply to the this modified version of the problem as well. However, the time complexity is no longer guaranteed to be O(logN). Look at the following examples.

The two cases mentioned below are easier to solve because the middle element is different from the first and the last elements and can help direct the binary search (although you’d get stuck with a 4 as the mid point further down the binary search).

The point being is that since duplicate elements are allowed here, it is possible to have a scenario where:

leftmost element == middle element == rightmost element

and when this scenario takes place, how do we decide what direction we need to move towards. There is no possible way for us to know the direction that can be ignored by the binary search algorithm. So, we would have to try and consider both as possible candidates and process them and in case all of the elements are the same in our array i.e. [4,4,4,4,4,4,4,4] then we would eventually end up processing each of the elements one by one.

Therefore, with a heavy heart 😢 we have to conclude that there is just no way to get a guaranteed O(logN) complexity algorithm on this question. The worst case time complexity of a modified version of the binary search algorithm we looked at above would be O(N).

Let us move on to the final question for this article and it is going to be a blockbuster one. Trust me!

Orderly Queue

Take a look at the problem statement here.

Let’s look at some of the possible string rotations first before getting to the solution. The string we will consider is baaca and K = 3 that means we can select any of the first three characters and then remove it from its location, add it to the very end and finally shift all the characters one position to the left to accommodate this new element in the end.

Assume the string has the following characters: a[0], a[1], a[2] … a[n-1] and we want to swap some position i (i >= 0 && i < n — 1) with position i+1, or swap a[i] and a[i+1]. The claim is that we can achieve this for any two adjacent elements in the string by using rotations on the string. e.g.:- Say the string consists of 5 characters and we want to swap a[2] and a[3] , here’s how we can achieve this with array rotations.

a[0], a[1], **a[2], a[3]**, a[4], a[5]     ROTATE around first element
a[1], **a[2], a[3]**, a[4], a[5], a[0]     ROTATE around first element
**a[2], a[3]**, a[4], a[5], a[0], a[1]     ROTATE around **second** element
**a[2],** a[4], a[5], a[0], a[1], **a[3]**     ROTATE around **first** element
a[5], a[0], a[1], **a[3], a[2],** a[4]     ROTATE around first element
a[0], a[1], **a[3], a[2],** a[4], a[5]     ROTATE around first element

You can try playing around with this idea, but essentially we can swap any two adjacent elements in the given string by performing multiple rotations in the manner shown above.

Since we can swap any two elements, we can perform Bubble Sort!

The bubble sort algorithm essentially involves comparison amongst adjacent elements for the purpose of bubbling up/down elements to their respective positions in the array.

Thus we have achieved swapping of chars a[2] and a[3] without disturbing ordering of other characters (similarly this can be done for any pair of adjacent indices).

Therefore, if K > 1 in the question, we can essentially perform the bubble sort algorithm by using rotations and eventually the smallest lexicographic string that we would get would be the original string sorted in ascending order.

What about when K = 1?

In this case we don’t get that much freedom in “choosing” which element to move to the back of the array. In this case we have to look at all of the possible rotations of the original string and return the one that is lexicographically the smallest one.

If you remember correctly, the number of rotations for a string of size N are N. So, when K = 1, we would have to look at all of the array’s rotations (remember the mod method or concat methods we discussed in the article to get all rotations?) and obtain the smallest one lexicographically.

Let’s look at the implementation even though it is a very small one.

Time Complexity: O(NlogN) because we are sorting the string for K > 1

Space Complexity: O(N) because if K = 1, then we create S+S which is O(N) space allocation.

That’s it for this article. Hope you had a fun time learning rotations in arrays and I hope you were able to grasp all of the concepts that we discussed here.

All Hail Coding…

Finding Shortest Paths using Breadth First Search

2018-07-20T00:00:00-07:00

Finding Shortest Paths using Breadth First Search

Do you know the amount of global air traffic in 2017? Do you know what the rise has been for air traffic over the past several years ? Well, lets look at some statistics.

Source: [https://www.statista.com/statistics/564769/airline-industry-number-of-flights/](https://www.statista.com/statistics/564769/airline-industry-number-of-flights/)

According to the International Civil Aviation Organization (ICAO), a record 4.1 billion passengers were carried by the aviation industry on scheduled services in 2017. And, the number of flights rose to 37 million globally in 2017.

That’s a lot of passengers and a lot of flights occupying the air space on a daily basis across the world. Since there are hundreds and thousands of these flights all around the globe, there are bound to be different routes with multiple stops in between from one place to another.

Every flight has a source and destination of its own and a standard economy seat price associated with it. Let’s leave out the fancy business class tickets and extra leg room and what not!

In such a scenario, it is too confusing to choose what flight would be the best one if we want to go from one place to another.

Let’s see the number of flight options StudentUniverse (provides discounts for students 😜) gives me from Los Angeles to New Delhi.

Every flight has a Details hyperlink with it, so we searched for that and found 119 total flights.

119 total flights are being offered. Then there appears a pop up on the website saying that there are other websites that might be offering similar flights at even cheaper rates. 😩

So many websites and uncountable flights for just a single source and destination.

As a developer, if I want to solve this problem, I would build a system to efficiently address the following queries:

Total number of destinations reachable (with a max number of stops) from my current location, and also list those destinations. One should keep their options open when they want to travel 😃.
It is a known fact (IMO 😉) that a route with multiple stops tends to be a cheaper alternative to direct flights. So, given a source and a destination, we may want to find routes with at least 2 or 3 stops.
Most importantly: What is the cheapest route from a given source to a given destination?
And…. We’ll come to this one in the end 🙈.

As you might guess, there would be potentially thousands of flights as the output of the first two queries. But we can certainly reduce that by providing some other criteria to lessen the output size. For the scope of this article, let us focus on these original queries themselves.

Modeling the Flight Network as a Graph

It’s pretty clear from the headline of this article that graphs would be involved somewhere, isn’t it?

Modeling this problem as a graph traversal problem greatly simplifies it and makes the problem much more tractable. So, as a first step, let us define our graph.

We model the air traffic as a:

directed
possibly cyclic
weighted
forest.

Directed ~ because every flight will have a designated source and a destination. These carry a lot of meaning.

Cyclic ~ because it is very possible to follow a bunch of flights starting from a given location and ending back at the same location.

Weighted ~ because every flight has a cost associated with it which would be the economy class flight ticket for this article.

And finally, a forest because we might have multiple connected components. It is not necessary that all the cities in the world have some sort of flight network between them. So, the graph can be disconnected, and hence a forest.

The vertices, V, would be the locations all over the world wherever there are working airports.

The edges, E, would be representative of all the flights constituting the air traffic. An edge from u --> v simply means you have a directed flight from the location / node u to v .

Sample flight network with the cost labelling for different flights.

Now that we have an idea about how to model the flight network as a graph, let us move on and solve the first common query that a user might have.

Total Number of Destinations Reachable

Who doesn’t like to travel?

As someone who likes to explore different places, you would want to know what all destinations are reachable from your local airport. Again, there would be additional criteria here to reduce the results of this query. But to keep things simple, we will simply try and find all the locations reachable from our local airport.

Now that we have a well defined graph, we can apply traversal algorithms to process it.

Starting off from a given point, we can use either Breadth First Search (BFS) or Depth First Search (DFS) to explore the graph or the locations reachable from the starting location within a maximum number of stops. Since this article is all about the breadth first search algorithm, let’s look at how we can use the famous BFS to accomplish this task.

We will initialize the BFS queue with the given location as the starting point. We then perform the breadth first traversal, and keep going until the queue is empty or until the maximum number of stops have been exhausted.

Note: If you are not familiar with the breadth first search or depth first search, I would recommend going through this article before continuing.

Let’s look at the code to initialize our graph data structure. We also need to look at how the BFS algorithm would end up giving us all the destinations reachable from a given source.

Some Random set of flights with source, destination and their prices.

Now that we have a good idea about how the graph is to be initialized, let’s look at the code for the BFS algorithm.

Performing bfs on the city of Los Angeles would give us the following destinations which are reachable:

{'Chicago', 'France', 'Ireland', 'Italy', 'Japan', 'New Delhi', 'Norway'}

That was simple, wasn’t it?

We will look at how we can limit the BFS to a maximum number of stops later on in the article.

In case we have a humongous flight network, which we would have in a production scenario, then we would not ideally want to explore all the reachable destinations from a given starting point.

This is a use case if the flight network is very small or pertains only to a few regions in the United States.

But, for a large network, a more realistic use case would be to find all the flight routes with multiple stops. Let us look at this problem in some more detail and see how we can solve it.

Routes with multiple stops

It is a well known fact that more often than not, for a given source and destination, a multi stop trip is cheaper than a direct, non-stop flight.

A lot of times we prefer the direct flight to avoid the layovers. Also because the multi-stop flights do tend to take a lot of time — which we don’t have.

However, if you don’t have any deadlines approaching and you want to save some bucks (and are comfortable with the multi-stop route that a lot of airlines suggest), then you might actually benefit a lot from something like this.

Also, you might get to pass through some of the most beautiful locations in the world with some of the most advanced airports which you can enjoy. So, that’s enough motivation as it is.

In terms of the graph model that we have been talking about, given a source and a destination, we need to come up with routes with 2 or more stops for a given source and destination.

As an end user, we might not want to see flights in this order for this query:

[A, C, D, B], 2 stops, $X  
[A, E, D, C, F, W, G, T, B], 8 stops, $M  
[A, R, E, G, B], 3 stops, $K  
[A, Z, X, C, V, B, N, S, D, F, G, H, B, 11 stops, $P

I know. Nobody in their right minds would want to go for a flight route with 11 stops. But the point I’m trying to make is that an end user would want symmetry. Meaning that they would want to see all the flights with 2 stops first, then all the flights with 3 stops and so on till maybe a max of, say, 5 stops.

But, all the flight routes with the same number of stops in between should be displayed together. That is a requirement we need to satisfy.

Let’s look at how we can solve this. So, given the graph of flight networks, a source S and a destination D, we have to perform a level order traversal and report flight routes from S --> D with at least 2 and at most 5 stops in between. This means we have to do a level order traversal until a depth of 7 from the start node S .

Have a look at the code for solving this problem:

This might not be the best way to go about solving this problem at scale — the largest memory constraint would be due to the nodes currently present in the queue.

Since every node or location can have thousands of flights to other destinations in the world, the queue could be humongous if we store actual flight data like this. This is just to demonstrate one of the use cases of the breadth first search algorithm.

Now, let us just focus on the traversal and look at the way it is done. The traversal algorithm is simple as it is. However, the entire space complexity of the level order traversal comes from the elements in the queue and the size of each element.

There are multiple ways to implement the algorithm. Also, each of them varies in terms of maximum memory consumed at any given time by the elements in the queue.

We want to see the maximum memory consumed by the queue at any point in time during the level order traversal. Before that, let’s construct a random flight network with random prices.

Now let us look at the implementation of level order traversal.

This above is the easiest and most straightforward implementation of the level order traversal algorithm.

With every node we add to the queue, we also store the level information and we push a tuple of (node, level) into the queue. So every time we pop an element from the queue, we have the level information attached with the node itself.

The level information for our use case would mean the number of stops from the source to this location in the flight route.

It turns out that we can do better as far as memory consumption of the program is concerned. Let us look at a slightly better approach to doing level order traversal.

The idea here is that we don’t store any additional information with the nodes being pushed into the queue. We use a None object to mark the end of a given level. We don’t know the size of any level before hand except for the first level, which just has our source node.

So, we start the queue with [source, None] and we keep popping elements. Every time we encounter a None element, we know that a level has ended and a new one has started. We push another None to mark the end of this new level.

Consider a very simple graph here, and then we will dry run this through the graph.

************************** LEVEL 0 begins  
level = 0, queue = [A, None]  
level = 0, pop, A, push, B, C, queue = [None, B, C]  

pop None ********************** LEVEL 1 begins  
push None  
level = 1, queue = [B, C, None]  
level = 1, pop, B, push, C, D, F, queue = [C, None, C, D, F]  
level = 1, pop, C, push, D, D (lol!), queue = [None, C, D, F, D, D]  

pop None ********************** LEVEL 2 begins  
push None  
level = 2, queue = [C, D, F, D, D, None] .... and so on

I hope this sums up the algorithm pretty well. This certainly is a neat trick to do level order traversal, keep track of the levels, and not encounter too much of a memory concern. This certainly reduces the memory footprint of the code.

Don’t get complacent now thinking this is a great improvement.

It is, but you should be asking two questions:

How big of an improvement is this?
Can we do better?

I will answer both of these questions now starting with the second question. The answer to that is Yes!

We can do one better here and completely do away with the need for the None in the queue. The motivation for this approach comes from the previous approach itself.

If you look closely at the dry run above, you can see that every time we pop a None, one level is finished and the other one is ready for processing. The important thing is that an entire next level exists in the queue at the time of popping of a None . We can make use of this idea of considering the queue size into the traversal logic.

Here is the pseudo code for this improved algorithm:

queue = Queue()  
queue.push(S)  
level = 0  
while queue is not empty {  
      size = queue.size()  
      // size represents the number of elements in the current level  
      for i in 1..size {  
          element = queue.pop()  
          // Process element here  
          // Perform a series of queue.push() operations here level += 1

And here is the code for the same.

The pseudo code is self explanatory. We essentially do away with the need for an extra None element per level and instead make use of the queue’s size to change levels. This would also lead to improvement over the last method, but how much?

Have a look at the following Jupyter Notebook to see the memory difference between the three methods.

We track the maximum size of the queue at any time by considering the sum of sizes of individual queue elements.
According to Python’s documentation, sys.getsizeof returns the object’s pointer or reference’s size in bytes. So, we saved almost 4.4Kb space (20224 — 15800 bytes) by switching to the None method from the original level order traversal method. This is just the memory savings for this random example, and we went only until the 5th level in the traversal.
The final method only gives an improvement of 16 bytes over the None method. This is because we got rid of just 4 None objects which were being used to mark the 4 levels (apart from the first one) that we processed. Each pointer’s size (pointer to an object) is 4 bytes in Python on a 32 bit system.

Now that we have all these interesting multi-path routes from our source to our destination and highly efficient level order traversal algorithms to solve it, we can look at a more lucrative problem to solve using our very own BFS.

What’s the cheapest flight route from my source to a given destination? This is something everybody would be instantly interested in. I mean who doesn’t want to save some bucks?

Shortest Path from a given source to destination

There’s not much description to give for the problem statement. We just need to find the shortest path and make the end user happy.

Algorithmically, given a weighted directed graph, we need to find the shortest path from source to destination. Shortest or cheapest would be one and the same thing from the point of the view of the algorithm.

We will not go into describing a possible BFS solution to this problem because such a solution would be intractable. Let us look at the graph below to understand why that is the case.

We say that BFS is the algorithm to use if we want to find the shortest path in an undirected, unweighted graph_._ The claim for BFS is that the first time a node is discovered during the traversal, that distance from the source would give us the shortest path.

The same cannot be said for a weighted graph. Consider the graph above. If say we were to find the shortest path from the node A to B in the undirected version of the graph, then the shortest path would be the direct link between A and B. So, the shortest path would be of length 1 and BFS would correctly find this for us.

However, we are dealing with a weighted graph here. So, the first discovery of a node during traversal does not guarantee the shortest path for that node. For example, in the diagram above, the node B would be discovered initially because it is the neighbor of A and the cost associated with this path (an edge in this case) would be 25 . But, this is not the shortest path. The shortest path is A --> M --> E --> B of length 10.

Breadth first search has no way of knowing if a particular discovery of a node would give us the shortest path to that node. And so, the only possible way for BFS (or DFS) to find the shortest path in a weighted graph is to search the entire graph and keep recording the minimum distance from source to the destination vertex.

This solution is not feasible for a huge network like our flight network that would have potentially thousands of nodes.

We won’t go into the details of how we can solve this. That is out of scope for this article.

What if I told you that BFS is just the right algorithm to find the shortest path in a weighted graph with a slight constraint ?

Constrained Shortest Paths

Since the graph we would have for the flight network is humongous, we know that exploring it completely is not really a possibility.

Consider the problem of shortest paths from the customer’s perspective. When you want to book a flight, these are the following options you ideally consider:

It shouldn’t be too long a flight.
It should be under your budget (Very Important).
It may have multiple stops but not more than K where K can vary from person to person.
Finally we have personal preferences which involve things like lounge access, food quality, layover locations, and average leg room.

The important point to consider here is the third one above: it may have multiple stops, but not more than K where K can vary from person to person.

A customer wants the cheapest flight route, but they also don’t want say 20 stops in between their source and destination. A customer might be okay with a maximum of 3 stops, or in extreme cases maybe even 4 — but not more than that.

We would want an application that would find out the cheapest flight route with at most K stops for a given source and destination.

This question from LeetCode has been the primary motivation for me to write this article. I strongly recommend going through the question once and not only relying on the snapshot above.

“Why would BFS work here?” one might ask. “This is also a weighted graph and the same reason for the failure of BFS that we discussed in the previous section should apply here.” NO!

The number of levels that the search would go to is limited by the value K in the question or in the description provided at the start of section. So, essentially, we would be trying to find the shortest path, but we won’t have to explore the entire graph as such. We will just go up to the level K.

From a real life scenario, the value of K would be under 5 for any sane traveler 😝.

Let us look at the pseudo-code for the algorithm:

def bfs(source, destination, K):  
      min_cost = dictionary representing min cost under K stops for each node reachable from source.       set min_cost of source to be 0 Q = queue()  
      Q.push(source)  
      stops = 0  
      while Q is not empty { size = Q.size  
           for i in range 1..size {  
                 element = Q.pop()                 if element == destination then continue for neighbor in adjacency list of element {  
                        if stops == K and neighbor != destination        then continue                        if min_cost of neighbor improves, update and add back to the queue.  
                }  
           }      
           stops ++   
      }

This again is level order traversal and the approach being used here is the one that makes use of the queue’s size at every level. Let us look at a commented version of the code to solve this problem.

Essentially, we keep track of the minimum distance of every node from the given source. The minimum distance for the source would be 0 and +inf for all others initially.

Whenever we encounter a node, we check if the current minimum path length can be improved or not. If it can be improved, that means that we have found an alternate path from source to this vertex with cheaper cost — a cheaper flight route until this vertex. We queue this vertex again so that locations and nodes reachable from this vertex on are updated (may or may not be) as well.

The key thing is this:

# No need to update the minimum cost if we have already exhausted our K stops.   
if stops == K and neighbor != dst:  
    continue

So we just popped the a node represented by element in the code and neighbor can either be a destination or a random other node. If we have already exhausted our K stops with the element being the Kth stop, then we shouldn’t process and update (possibly) the minimum distance for neighbor. This would violate our maximum K stops condition in that case.

As it turns out, I solved this problem originally using Dynamic Programming and it took around 165ms to run on the LeetCode platform. I ran using BFS and it was blazing fast with just 45ms to execute. Motivation enough to write this article for you guys.

I hope you were able to derive some benefit from this article on Breadth First Search and some of its applications. The major focus was to showcase its application to shortest paths in a weighted graph under some constraints 😃.

Recursion Demystified

2018-07-12T00:00:00-07:00

Crazy, isn’t it ?

Well, I hope that by the end of this article you will feel much more confident about what recursion is and mainly, how we can come up with a recursive solution to a problem.

What is Recursion?

How do you explain recursion to a 4 year old? This is a pretty famous interview question, and there are loads of answers available on the web. We won’t answer this question as it is too mainstream.

If you are as clever as I am 🤓🤓, you would explain recursion to someone one year younger than you. Have them explain recursion to someone one year younger than them. Continue until you have a 5 year old explaining recursion to a 4 year old. Done. [Source: reddit].

In programming terms, recursion is

A function calling itself.

The above function does no useful work as such, but it does demonstrate recursion. The recursive relation above would be

T(N) = T(N - 1) + O(1)

This simply means that the execution for the call to random_function(n) cannot proceed until the call to random_function(n-1) is completed and so on.

Essentially, we delay the execution of the current state of the function until another call to the same function has completed and returned it’s result.

The compiler keeps on saving the state of the function call now and then moves onto the next function call and so on. So, the compiler saves function states onto a stack and uses that for computations and backtracking.

Recursion stack of a set of function calls.

Essentially, if a problem can be broken down into similar subproblems which can be solved individually, and whose solutions can be combined together to get the overall solution, then we say that there might exist a recursive solution to the problem.

Instead of clinging to this seemingly old definition of recursion, we will look at a whole bunch of applications of recursion. Then hopefully things will be clear.

Factorial of a Number

Let us see how we can find out the factorial of a number. Before that, let’s see what the factorial of a number represents and how it is calculated.

factorial(N) = 1 * 2 * 3 * .... * N - 1 * N

Simply put, the factorial of a number is just the product of terms from 1 to the number N multiplied by one another.

We can simply have a for loop from 1 to N and multiply all the terms iteratively and we will have the factorial of the given number.

But, if you look closely, there exists an inherent recursive structure to the factorial of a number.

factorial(N) = N * factorial(N - 1)

It’s like offloading the computation to another function call operating on a smaller version of the original problem. Let’s see how this relation would unfold to verify if the solution here matches the one provided by the for loop.

Showing the steps from top to bottom for the factorial recursive function.

Verification that the recursive function defined produces the correct result.

So, it is clear from the two figures above that the recursive function that we defined earlier,

factorial(N) = N * factorial(N - 1)

is indeed correct. Have a look at the Python code snippet used to find the factorial of a function, recursively.

This example was pretty simple. Let us consider a slightly bigger but standard example to demonstrate the concept of recursion.

Fibonacci Sequence

You must be already familiar with the famous fibonacci sequence. For those of you who have’t heard about this sequence or seen an example before, lets have a look.

1 1  2   3     5           8                       13 .....

Let us look at the formula for calculating the n^th fibonacci number.

F(n) = F(n - 1) + F(n - 2)  
where F(1) = F(2) = 1

Clearly, this definition of the fibonacci sequence is recursive in nature, since the n^th fibonacci number is dependent upon the previous two fibonacci numbers. This means dividing the problem into smaller subproblems, and hence recursion. Have a look at the code for this:

Every recursive problem must have two necessary things:

The recurrence relation defining the states of the problem and how the main problem can be broken down into smaller subproblems. This also includes the base case for stopping the recursion.
A recursion tree that showcases the first few, if not all calls to the function under consideration. Have a look at the recursion tree for the fibonacci sequences’ recursive relation.

Recursion tree showing the sequence of calls for the fibonacci recurrence relation.

The recursion tree shows us that the results obtained from processing the two subtrees of the root N can be used to compute the result for the tree rooted at N. Similarly for other nodes.

The leaves of this recursion tree would be fibonacci(1) or fibonacci(2) both of which represent the base cases for this recursion.

Now that we have a very basic grasp of recursion, what a recurrence relation is, and the recursion tree, let’s move onto something more interesting.

Examples!

I strongly believe in solving umpteen number of examples for any given topic in programming to become a master of that topic. The two examples we considered (Factorial of a number and the Fibonacci sequence) had well defined recurrence relations. Let us look at a few examples where the recurrence relation might not be so obvious.

Height of a Tree

To keep things simple for this example, we will only consider a binary tree. So, a binary tree is a tree data structure in which each node has at most two children. One node of the tree is designated as the root of the tree, for example:

A Tree rooted at ‘A’ with its height and the corresponding path highlighted.

Let’s define what we mean by the height of the binary tree.

Height of the tree would be the length of the longest root to leaf path in the tree.

So, for the example diagram displayed above, considering that the node labelled as A as the root of the tree, the longest root to leaf path is A → C → E → G → I . Essentially, the height of this tree is 5 if we count the number of nodes and 4 if we just count the number of edges on the longest path.

Now, forget about the entire tree and just focus on the portions highlighted in the diagram below.

Tree rooted at node A and it’s two subtrees with their respective heights.

The above figure shows us that we can represent a tree in the form of its subtrees. Essentially, the structure to the left of node A and the structure to the right of A is also a binary tree in itself, just smaller and with different root nodes. But, they are binary trees nonetheless.

What information can we get from these two subtrees that would help us find the height of the main tree rooted at A ?

If we knew the height of the left subtree, say h1, and the height of the right subtree, say h2, then we can simply say that the maximum of the two + 1 for the node A would give us the height of our tree. Isn’t that right?

Formalizing this recursive relation,

height(root) = max(height(root.left), height(root.right)) + 1

So, that’s the recursive definition of the height of a binary tree. The focus is on binary here, because we used just two children of the node root represented by root.left and root.right. But, it is easy to extend this recursive relation to an n-ary tree. Let’s take a look at this in code.

The problem here was greatly simplified because we let recursion do all the heavy lifting for us. We simply used optimal answers for our subproblems to find a solution to our original problem.

Let’s look at another example that can be solved on similar lines.

Number of Nodes in a Tree

Here again, we will consider a binary tree for simplicity, but the algorithm and the approach can be extended to any kind of tree essentially.

The problem is itself very self explanatory. Given the root of a binary tree, we need to determine the total number of nodes in the tree. This question and the approach we will come up with here are very similar to the previous one. We just have to make minuscule changes and we will have the number of nodes in the binary tree.

Take a look at the diagram below.

The diagram says it all. We already know that a tree can be broken down into smaller subtrees. Here again, we can ask ourselves,

What information can we get from these two subtrees that would help us find the number of nodes in the tree rooted at A?

Well, if we knew the number of nodes in the left subtree and the number of nodes in the right subtree, we can simply add them up and add one for the root node and that would give us the total number of nodes.

Formalizing this we get,

number_of_nodes(root) = number_of_nodes(root.left) + number_of_nodes(right) + 1

If you look at this recursion and the previous one, you will find that they are extremely similar. The only thing that is varying is what we do with the information we obtained from our subproblems and how we combined them to get some answer.

Now that we have seen a couple of easy examples with a binary tree, let’s move onto something less trivial.

Merge Sort

Given an array of numbers like

4 2 8 9 1 5 2

we need to come up with a sorting technique that sorts them either in ascending or descending order. There are a lot of famous sorting techniques out there for this like Quick Sort, Heap Sort, Radix Sort and so on. But we are specifically going to look at a technique called the Merge Sort.

It’s possible that a lot of you are familiar with the Divide and Conquer paradigm, and this might feel redundant. But bear with me and read on!

The idea here is to break it down into subproblems.

That’s what the article is about right ? 😛

What if we had two sorted halves of the original array. Can we use them somehow to sort the entire array?

That’s the main idea here. The task of sorting an array can be broken down into two smaller subtasks:

sorting two different halves of the array
then using those sorted halves to obtain the original sorted array

Now, the beauty about recursion is, you don’t need to worry about how we will get two sorted halves and what logic will go into that. Since this is recursion, the same method call to merge_sort would sort the two halves for us. All we need to do is focus on what we need to do once we have the sorted haves with us.

Let’s go through the code:

At this point, we trusted and relied on our good friend recursion and assumed that left_sorted_half and right_sorted_half would in fact contain the two sorted halves of the original array.

So, what next?

The question is how to combine them somehow to give the entire array.

The problem now simply boils down to merging two sorted arrays into one. This is a pretty standard problem and can be solved by what is known as the “two finger approach”.

Take a look at the pseudo code for better understanding.

let L and R be our two sorted halves.   
let ans be the combined, sorted array

l = 0 // The pointer for the left half  
r = 0 // The pointer for the right half  
a = 0 // The pointer for the array ans

while l < L.length and r < R.length {  
    if L[l] < R[r] {  
        ans[a] = L[l]  
        l++  
    } else {  
        ans[a] = R[r]  
        r++  
    }  
}

copy remaining array portion of L or R, whichever was longer, into ans.

Here we have two pointers (fingers), and we position them at the start of the individual halves. We check which one is smaller (that is, which value pointed at by the finger is smaller), and we add that value to our sorted combined array. We then advance the respective pointer (finger) forward. In the end we copy the remaining portion of the longer array and add it to the back of the ans array.

So, the combined code for merge-sort is as follows:

We will do one final question using recursion and trust me, it’s a tough one and a pretty confusing one. But before moving onto that, I will iterate the steps I follow whenever I have to think of a recursive solution to a problem.

Steps to come up with a Recursive Solution

Try and break down the problem into subproblems.

Source: [https://www.weheartswift.com/compute-2-power-n/](https://www.weheartswift.com/compute-2-power-n/).

Once you have the subproblems figured out, think about what information from the call to the subproblems can you use to solve the task at hand. For example, the factorial of N — 1 to find the factorial of N , height of the left and right subtrees to find the height of the main tree, and so on.

Keep calm and trust recursion! Assume that your recursive calls to the subproblems will return the information you need in the most optimal fashion.

The final step in this process is actually using information we just got from the subproblems to find the solution to the main problem. Once you have that, you’re ready to code up your recursive solution.

Now that we have all the steps lined up, let’s move on to our final problem in this article. It’s called Sum of Distances in a Tree.

Sum of Distances in a Tree

Let’s look at what the question is asking us to do here. Consider the following tree.

Example tree showing the expected output of our program for the various nodes.

In the example above, the sum of paths for the node A (the number of nodes on each path from A to every other vertex in the tree) is 9. The individual paths are mentioned in the diagram itself with their respective lengths.

Similarly, consider the sum of distances for the node C.

C --> A --> B (Length 2)  
C --> A (Length 1)  
C --> D (Length 1)  
C --> E (Length 1)  
C --> D --> F (Length 2)  
Sum of distances (C) = 2 + 1 + 1 + 1 + 2 = 7

This is known as the sum of distances as defined for just a single node A or C. We need to calculate these distances for each of the nodes in the tree.

Before actually solving this generic problem, let us consider a simplified version of the same problem. It says that we just need to calculate the sum of distances for a given node, but we will only consider the tree rooted at the given node for calculations.

So, for the node C, this simplified version of the problem would ask us to calculate:

C --> D (Length 1)  
C --> E (Length 1)  
C --> D --> F (Length 2)  
Simplified Sum of Distances (C) = 1 + 1 + 2 = 4

This is a much simpler problem to tackle recursively and would prove to be useful in solving the original problem.

Consider the following simple tree.

The simple tree we would consider for now. Also mentioned are two values we would be computing for every node.

The nodes B and C are the children of the root (that is, A).

We are trying to see what information can we use from subproblems (the children) to compute the answer for the root A .

Note: here we simply want to calculate the sum of paths for a given node X to all its successors in its own subtree (the tree rooted at the node X).

There are no downwards going paths from the node B, and so the sum of paths is 0 for the node B in this tree. Let’s look at the node C . So this node has 3 different successors in F, D and E . The sum of distances are as follows:

C --> D (Path containing just 1 edge, hence sum of distances = 1)  
C --> D --> F (Path containing 2 edges, hence sum of distances = 2)  
C --> E (Path containing just 1 edge, hence sum of distances = 1)

The sum of all the paths from the node C to all of its decedents is 4, and number of such paths going down is 3.

Note the difference here. The sum_of_distances here counts the number of edges in each path — with each edge repeating multiple times, probably because of their occurrence on different paths — unlike number_of_paths , which counts, well, the number of paths 😝.

If you look closely, you will realize that the number of paths going down is always going to be the number of nodes in the tree we are considering (except the root). So, for the tree rooted at C, we have 3 paths, one for the node D, one for E, and one for F. This means that the number of paths from a given node to the successor nodes is simply the total number of descendent nodes since this is a tree. So, no cycles or multiple edges.

Now, consider the node A. Let us look at all the new paths that are being introduced because of this node A. Forget the node B for now and just focus on the child node C corresponding to A. The new sets of paths that we have are:

A --> C (Path containing just 1 edge, hence sum of distances = 1)  
A --> (C --> D)    (Path containing 2 edges, hence sum of distances = 2)  
A --> (C --> E)    (Path containing 2 edges, hence sum of distances = 2)  
A --> (C --> D --> F) (Path containing 3 edges, hence sum of distances = 3)

Except for the first path A → C, all the others are the same as the ones for the node C, except that we have simply changed all of them and incorporated one extra node A.

Sum of distances for the node A along with contribution from the node C.

If you look at the diagram above you will see a tuple of values next to each of the nodes A, B, and C.

(X, Y) where   
X is the number of paths originating at that node and going down to the decedents.   
Y is the sum of distances for the tree rooted at the given node. 

Since the node B doesn’t have any further children, the only path it is contributing to is the path A --> B to A's tuple of (5, 9) above. So let’s talk about C.

C had three paths going to its successors. Those three paths (extended by one more node for A) also become three paths from A to its successors, among others.

N-Paths[A] = (N-Paths[C] + 1) + (N-Paths[B] + 1)

That is the exact relation we are looking for as far as the number of paths (= number of successor nodes in the tree) are concerned. The 1 is because of the new path from the root to it’s child, that is A --> C in our case.

N-Paths[A] = 3 + 1 + 0 + 1 = 5

As far as the sum of distances is concerned, take a look at the diagram and the equations we just wrote. The following formula becomes very clear:

Sum-Dist[A] = (N-Paths[C] + 1 + Sum-Dist[C]) + (N-Paths[B] + 1 + Sum-Dist[B])Sum-Dist[A] = (3 + 1 + 4 + 0 + 1 + 0) = 9

The main thing here is N-Paths[C] + Sum-Dist[C] . We sum these up because all of the paths from C to its descendants ultimately become the paths from A to its descendants — except that they originate at A and go through C, and so each of the path lengths are increased by 1. There are N-Paths[C] paths in all originating from C and their total length is given by Sum-Dist[C] .

Hence the tuple corresponding to A = (5, 9). The Python code for the algorithm we discussed above is as follows:

The Curious Case of the Visited Dictionary :/

If you look at the code above closely, you’ll see this:

# Prevents the recursion from going into a cycle.
self.visited[vertex] = 1

The comment says that this visited dictionary is for preventing the recursion from entering a cycle.

If you’ve paid attention til now, you know that we are dealing with a tree here.

The definition of a tree data structure doesn’t allow cycles to exist. If a cycle exists in the structure, then it is no longer a tree, it becomes a graph. In a tree, there is exactly one path between any two pair of vertices. A cycle would mean there is more than one path between a pair of vertices. Look at the figures below.

Tree vs Graph. Showing the cycle.

The structure on the left is a tree. It has no cycles in it. There is a unique path between any two vertices.

The structure on the right is a graph, there exists a cycle in the graph and hence there are multiple paths between any pair of vertices. For this graph, it so happens that any pair of vertices have more than one path. This is not necessary for every graph.

Almost always, we are given the root node of the tree. We can use the root node to traverse the entire tree without having to worry about any cycles as such.

However, if you’ve read the problem statement clearly, it does not state anything about root of the tree.

That means that there is no designated root for the tree given in the question. This could mean that a given tree can be visualized and processed in so many different ways depending upon what we consider as the root. Have a look at multiple structures for the same tree but with different root nodes.

Multiple orientations of the same tree with different roots.

So many different interpretations and parent child relationships are possible for a given unrooted tree.

So, we start with the node 0 and do a DFS traversal of the given structure. In the process we fix the parent child relationships. Given the edges in the problem, we construct an undirected graph-like structure which we convert to the tree structure. Taking a look at the code should clear up some of your doubts:

Every node would have one parent. The root won’t have any parent, and the way this logic is, the node 0 would become the root of our tree. Note that we are not doing this process separately and then calculating the sum of distances downwards. Given a tree, we were trying to find, for every node, the simplified sum of distances for the tree rooted at that node.

So, the conversion from the graph to the tree happens in one single iteration along with finding out the sum of distances downwards for each and every node.

I posted the code again so that the visited dictionary makes much more sense now. So, one single recursion doing all that for us. Nice!

Bringing it all together

Now that we have our tree structure defined, and also the values of sum of distances going downward defined for us, we can use all of this information to solve the original problem of Sum of Distances in a Tree.

How do we do that? It’s best to explain this algorithm with the help of an example. So we will consider the tree below and we will dry run the algorithm for a single node. Let’s have a look at the tree we will be considering.

The tree we will be considering for our explanation moving on.

The node for which we want to find the sum of distances is 4. Now, if you remember the simpler problem we were trying to solve earlier, you know that we already have two values associated with each of the nodes:

distances_down Which is the sum of distances for this node while only considering the tree beneath_._
number_of_paths_down which is the number of paths / nodes in the tree rooted at the node under consideration.

Let’s look at the annotated version of the above tree. The tree is annotated with tuples (distances_down, number_of_paths_down) .

Example tree with annotated values for all nodes.

Let’s call the value we want to compute for each node as sod which means sum of distances, which is what the question originally asks us to compute.

Let us assume that we have already computed the answer for the parent node of 4 in the diagram above. So, we now have the following information for the node labelled 2 (the parent node) available:

(sod, distances_down, number_of_paths_down) = (13, 4, 3)

Let’s rotate the given tree and visualize it in a way where 2 is the root of the tree essentially.

Rotated tree showing the tree rooted at 4 should be removed.

Now, we want to remove the contribution of the tree rooted at 4 from sod(2). Let us consider all of the paths from the parent node 2 to all other nodes except the ones in the tree rooted at 4 .

--> 5 (1 edge)  
--> 1 (1 edge)  
--> 1 -->7 (2 edges)  
--> 1 --> 7 --> 9 (3 edges)  
--> 1 --> 7 --> 10 (3 edges)Number of nodes considered = 6  
Sum of paths remaining i.e. sod(2) rem = 1 + 1 + 2 + 3 + 3 = 10

Let’s see how we can use the values we already have calculated to get these updated values.

* N = 8 (Total number of nodes in the tree. This will remain the same for every node. )  
* sod(2) = 13

* distances_down[4] = 1  
* number_of_paths_down[4] = 1

* (distances_down[4] does not include the node 4 itself)  
N - 1 - distances_down[4] = 8 - 1 - 1 = 6

* sod(2) - 1 - distances_down[4] - number_of_paths_down[4] = 13 - 1 - 1 - 1 = 10

If you remember this from the function we defined earlier, you will notice that the contribution of a child node to the two values distances_down and number_of_paths_down is n_paths + 1 and n_paths + s_paths + 1 respectively. Naturally, that is what we subtract to obtain the remaining tree.

sod(4) represents the sum of edges on all the paths originating at the node 4 in the tree above. Let’s see how we can find this out using the information we have calculated till now.

distances_down[4] represents the answer for the tree rooted at the node 4 but it only considers paths going to its successors, that is all the nodes in the tree rooted at 4. For our example, the successor of 4 is the node 6. So, that will directly add to the final answer. Let’s call this value own_answer . Now, let’s account for all the other paths.

4 --> 2 (1 edge)  
4 --> 2 --> 5 (1 + 1 edge)  
4 --> 2 --> 1 (1 + 1 edge)  
4 --> 2 --> 1 -->7 (1 + 2 edges)  
4 --> 2 --> 1 --> 7 --> 9 (1 + 3 edges)  
4 --> 2 --> 1 --> 7 --> 10 (1 + 3 edges)  
own_answer = 1

sod(4) = 1 + 1 + 2 + 2 + 3 + 4 + 4 = 17

sod(4) = own_answer + (N - 1 - distances_down[4]) + (sod(2) - 1 - distances_down[4] - number_of_paths_down[4]) = 1 + 6 + 10 = 17

Source: [https://giphy.com/gifs/angry-confused-love-life-FX1ZGci9qPM3u](https://giphy.com/gifs/angry-confused-love-life-FX1ZGci9qPM3u)

Before you go bonkers and start doing this, let’s look at the code and bring together all of the things we discussed in the example above.

The recursive relation for this portion is as follows:

Recursive relation for Sum of Distances, in its totality.

Did I just see MEMOIZATION in the code?

Yes, indeed you did!

Consider the following example tree:

Example of a skewed tree, also recursive calls for the vertex 5.

The question asks us to find the sum of distances for all the nodes in the given tree. So, we would do something like this:

for i in range(N):  
    ans.append(find_distances(N))

But, if you look at the tree above, the recursive call for the node 5 would end up calculating the answers for all the nodes in the tree. So, we don’t need to recalculate the answers for the other nodes again and again.

Hence, we end up storing the already calculated values in a dictionary and use that in further calculations.

Essentially, the recursion is based on the parent of a node, and multiple nodes can have the same parent. So, the answer for the parent should only be calculated once and then be used again and again.

If you’ve managed to read the article this far (not necessarily in one stretch 😛), you’re awesome 😃.

Source: [http://doodlecats.com/youre-awesome](http://doodlecats.com/youre-awesome).

If you found this article helpful, share as much as possible and spread the 💛. Cheers!

Change the signs, a Codechef Problem

2018-06-01T00:00:00-07:00

If you’re a competitive programmer like I am, one of the best feelings in the world is seeing your program getting accepted on first try on one of the most famous programming platforms, CodeChef.

The CodeChef May 2018 Long Challenge ended about an hour ago, and I decided to write this article as a post describing one of the questions in the competition.

Without wasting any more time, let’s get to it.

Unravelling the Problem Statement

Let’s look at some examples to better understand what the problem statement is asking for.

Consider the following number sequence.

4 3 1 2

Now the question asks us to perform a certain operation (possibly 0 times, leaving the sequence unchanged). We can negate a certain subsequence of numbers and get a new sequence.

-4 3 1 2  
4 -3 1 -2  
4 3 -1 2  
4 3 1 -2  
-4 -3 1 2 etc.

The question says that the resulting sequence should satisfy the following constraint:

Note: The sum of elements of any substring with length greater than 1 is strictly positive.

Clearly, the following sequences are not valid:

-4 3 1 2  
4 -3 1 -2   
4 3 1 -2   
-4 -3 1 2   
-4 -3 -1 -2  
4 3 -1 -2

We only have 2 valid subsequences that can be obtained by performing the operation mentioned above. Note that we haven’t written down all the possible subsequences. That would be 2^n, that is 16 in this case, because for every number we have two options. Either to negate it, or not.

So the two valid sequences are:

4 3 1 2

and

4 3 -1 2

The original sequence would always be one of the valid sequences as all the numbers in it are positive.

Now the question asks us to find the sequence with the minimum sum. So for the example we have considered, the sequence required would be 4 3 -1 2 .

Would Greedy Work?

A greedy approach in this question would be that if it is possible to negate a number while satisfying the given constraints, then we should negate that number. This approach however, would not always give the right results. Consider the following example.

4 1 3 2

Here, it is possible to have these three valid sets of numbers:

4 1 3 2           4 -1 3 2           4 1 3 -2

Clearly, both the numbers 2 and 1 can be negated. But not both of them at the same time. If we negate a number greedily — that is, if a number can be negated, then we negate it — then it is possible that we might end up negating the number 1. Then you won’t be able to negate the number 2. This would give us a suboptimal solution.

So this Greedy approach would not work here. We have to try out a specific choice of whether to negate or not for a number and see what choice gives us the optimal solution.

This smells like Dynamic Programming.

Good ol’ Dynamic Programming

One of the most interesting algorithmic techniques out there, and possibly one of the most dreaded, is dynamic programming. This is the technique we are going to use to solve this particular problem.

Two of the most important steps in any dynamic programming problem are:

Identifying the recurrent relation.
Figuring out what to memoize.

The DP-based approach here is divided into two basic parts.

One is the main recursion that we use to find out the minimum sum of the final set. Note, the dynamic programming is not directly used to obtain the final set, just the sum of the final set of numbers. So our dynamic programming approach would correctly find out the sum for the example given above as 8. 4 + 3 + (-1) + 2 = 8 .
What we actually need is the final modified set of numbers where some (possibly none) of the numbers are negated. We use the concept of a parent pointer and backtracking to find out the actual set of numbers.

Let’s move onto our recursion relation for our dynamic programming approach.

Before describing the recursive relation an important observation to make here is that if a number has been negated, then any adjacent number to it can not be negative. That is, two adjacent numbers cannot be negative as that would give a substring of length 2 whose sum is negative, and that is not allowed according to the question.

For the recurrence relation, we need two variables. One is the index number of where we are in the array, and one is a boolean value that tells us if the previous number (one left to the previous number) is negated or not.

So if the current index is i, then the boolean value would tell us if the number at i — 2 was negated or not. You will know the importance of this boolean variable in the next paragraph.

We need to know in O(1) if a number can be negated or not. Since we are following a recursion with memoization-based solution, whenever we are at an index i in the recursion, we are sure that the numbers to the right (i+ 1 onwards) have not been processed up to this point. This means that all of them are still positive.

The choice of whether the number at index i can be negated is dependent upon the right hand side (if there is one) and the left hand side (if there is one). The right hand side is easy. All we need to check is if

number[i] < number[i + 1]

because if this is not true, then adding these two would give a negative value for the substring [i, i + 1] thus making it an invalid operation.

Now comes the tricky part. We need to see if negating the number at i will cause a substring of negative sum to the left or not. When we reach the index i in our recursion, we have already processed the numbers before it, and some might have been negated as well.

So say we have this set of numbers 4 1 2 1 and we had negated the first 1 and we are now processing the last number ( 1 ).

4 -1 2 [1]

The last number in square brackets is the one we are processing right now. As far as the right hand side is concerned, since there is none, we can negate it. We need to check if negating this 1 at index 3 (0 based indexing) would cause any substring to the left of ≤ 0 sum. As you can see, it will produce such a substring.

-1 2 -1

This substring would have a 0 sum, and that is invalid according to the question. After negating a subsequence of numbers, the substrings in the final set should have a sum which is strictly positive. All the substrings of length > 1.

We cannot apply the following approach here directly:

if number[i] < number[i - 1], then it is good to go on negation.

because, although 1 < 2 , if we negate that last 1 as well we will have an invalid set of numbers as seen above. So this simple approach or check won’t work here.

Here comes the boolean variable which tells us if, given an index i, the number at i — 2 was negated or not. Consider the two scenarios.

Yes, the number at index i — 2 was negated like in the example just showcased. In that case, negation of the number at i — 2 would have a capacity reduction for number at i — 1. In the example 4 1 2 1 , negating the 1 at index 1(0 based indexing) would reduce the capacity of the number 2 (at index 2) by 1. We refer to remaining values of numbers as capacities here. We need to consider this reduced capacity when performing the check to see if a number can be negated or not.

number[i] < reducedCapacityOfNumberAt(i - 1)

In case the number at index i — 2 wasn’t negated, the number at i — 1 is at it’s full capacity. The simple check

number[i] < number[i - 1]

would be enough to see if we can negate the number at index i .

Let’s look at the code for the recursion containing all the ideas discussed above.

That’s all nice and dandy. But, this is just recursion, and the heading says dynamic programming. That means there would be overlapping subproblems. Let us look at the recursion tree to see if there are any.

As you can see, there are overlapping subproblems in the recursion tree. That is why we can use memoization.

The memoization is as simple as:

""" This comes at the top. We check if the state represented by the tuple of the index and the boolean variable is already cached """
if(memo[i][is_prev_negated] != INF)   
{  
    return memo[i][is_prev_negated];  
}  

# ...... CODE  

# Cache the minimum sum from this index onwards.  
memo[i][is_prev_negated] = min(pos, neg);

# The parent pointer is used for finding out the final set of #s  
parent[i][is_prev_negated] = min(pos, neg) == pos ? 1 : -1;

As pointed out earlier, this recursive approach would return the minimum sum of the set of numbers possible after making the valid set of modifications to them.

The question, however, asks us to actually print the final set of numbers that gives the minimum sum after making such modifications. For that, we need to use a parent pointer that would tell us at every index and boolean variable is_prev_negated ’s value as to what optimal action was taken.

parent[i][is_prev_negated] = min(pos, neg) == pos ? 1 : -1;

So we simply store 1 or -1 depending upon if negating the number at index i (if possible!) gave us the minimum sum or if choosing to ignore it gave the minimum sum.

Backtracking

Now comes the part where we backtrack to find the solution to our original problem. Note that the decision for the very first number is what propagates the recursion further. If the first number was negated, the second number would be positive and the third number’s decision can be found using parent[2][true]. Similarly, if the first number wasn’t negated, then we move onto the second number and it’s decision can be found using parent[1][false] and so on. Let’s look at the code.

A Better Approach

If you take a look at the space complexity of the solution suggested, you will see that it’s a 2 dimensional dynamic programming solution because the state of the recursion is represented by two variables i.e. the index i representing what number of the array we are considering and then the boolean variable is_prev_negated . So the space complexity and the time complexity would be O(n*2) which is essentially O(n).

However, there is a slightly better approach as well to solving this problem which involves using a 1 dimensional dynamic programming based solution.

Essentially, the boolean variable is_prev_negated is helping us to decide if we can negate a given number at index i or not as far as the left hand side of the array is concerned i.e. all the numbers from 0 .. i-1 because the right hand side is anyways safe as all the numbers on that side are positive (as the recursion hasn’t reached them yet).

So for the right hand side we simply checked the number at i+1 but for the left hand side of index i we had to make use of the boolean variable is_prev_negated .

It turns out, that we can simply skip this boolean variable altogether and simply look ahead to decide if a number can be negated or not. Which simply means if you are at an index i, you check if that element along with the element at i+2 have the capacity to swallow the element at i+1 i.e.

numbers[i] + numbers[i+2] >= numbers[i+1  (SWALLOW)

If there is a such a possibility, then we directly jump to i+3if we negate element at i because element at i+1 and i+2 both can’t be negative in such a scenario.

In case the swallow condition is not satisfied and we end up negating the number at index i , then we would jump to index i+2 because in any case, two consecutive numbers cannot be negated. So if the number at i was negated, then the number at i+1 has to be positive. The swallow check is to see if the number at i+2 would definitely have to be positive or if we can exercise the choice of whether to negate or not there.

Have a look at the code for a better understanding.

Hence, just a single variable i.e. the index is used to define the state of the recursion. So the time and space complexity, both got reduced to half of what they were in the previous solution.

I hope you were able to grasp the working of the algorithm described above and how the dynamic programming technique fits into this problem. I think it’s an interesting problem, because you not only have to use dynamic programming but also the concept of parent pointer to retrace the steps through the optimal solution and get the answer required in the question.

I need a haircut

2018-05-15T00:00:00-07:00

“Should I get a haircut?”

“I think it’s been difficult to manage them lately and its too hot outside. I think I should get a haircut done.”

“Let me go to the nearby XYZ saloon and get this over with.”

The Walk

As I approach the salon, 1000 questions flood my head.

What if they cut my hair too short ?

What if they aren’t equal at the end of it ?

What if they use some weird oil on my head ?

Will they clean the razor before using it on my skin ?

Will they wash my hair afterwards ?

Will they understand exactly how I want my haircut to be ?

What if an inexperienced barber mistakenly cuts my skin ?

The Wait

As usual, there’s a huge line in waiting. Ahhh, the dreaded wait.

As I wait patiently for my turn to come, I constantly think about whom I want to cut my hair.

“Hmmmm, he looks experienced?”

“Oh no! he’s just not focussed”

“His work looks messy”

“Too much talking. Nah!”

“I think this is the perfect one. He’s listening to the customer and is taking his time”

But I almost never get the one I want. Never!

And, its my turn

And finally, they call out “Next!” and I know its my turn.

“Oh No. This is exactly the person I didn’t want me to work on my precious hair. He’ll surely spoil it.”

I’m seated on the hot seat and the million dollar question comes up.

“Sir, How would you like your hair cut?”

And my reply which has been the same for over a decade now is

“Don’t cut them too short, I want the length to be equal on all sides. Nothing fancy”

Somehow, nobody seems to grasp this perfectly. What they instead hear is

“Medium haircut”

Source: [https://smartblogger.com/fear-openings/](https://smartblogger.com/fear-openings/)

Why do they forget the part where they had to keep all my hair equal ?

Trimmer or Scissors

Now, the next scariest thing when the barber starts off is whether they will use the good old scissors or the dreaded trimmer.

The trimmer does the job very quickly, but sometimes too quickly. And the scissors, well, it does take time but they’re the best if you ask me.

As usual, the barber decides to use a trimmer. :((

Now this is the moment where my heart starts pounding.

“The trimmer is coming close.”

“What if he makes a deep cut. There’s no going back.”

“Buckle up Sachin! This is it.”

Source: [https://www.thoughtco.com/using-regular-motor-oil-chainsaw-bar-1342724](https://www.thoughtco.com/using-regular-motor-oil-chainsaw-bar-1342724)

The barber brings a small trimmer towards me. But, what I see is this.

Source: [https://makeameme.org/meme/phew-thank-goodness](https://makeameme.org/meme/phew-thank-goodness)

Aaaaaaaaand. Zap!

“Oh GOD, did he cut it too short?”

Sometimes, its a yes and I feel as if I’m scarred for life.

Sometimes, its a no and I’m like “Phew!” . But the haircut session is not over yet. Things can still go bad.

After about 10 minutes of using the trimmer to butcher my hair, the barber decides to finally use a scissor and I feel a sigh of relief.

Source: [https://www.pinterest.com/pin/748301294306046836/?lp=true](https://www.pinterest.com/pin/748301294306046836/?lp=true)

The most peaceful time during the entire session is when the barber uses the scissors. I feel at peace. At least there won’t be anything sudden and they would cut my hair carefully.

It’s not like with trimmers. One mistake. One shake up. And a patch of my hair is gone. :/ :/

And then there’s no going back.

Anyways. Over the years I’ve learnt to tell the barber I want my hair cut to be medium which is not usually what I want. The reason is that if they cut it too short initially and not not according to what I want the hair can’t grow back instantly right ?

So I tell them to cut it medium initially and I tell them to shorten it again after they ask me if the hair cut looks good. 😛😛

It’s done

Finally, the hair cut is done. And I feel like its a disaster. As usual.

I am never happy with the hair cut. I have to put up a fake smile though.

Source: [https://www.lifesitenews.com/blogs/as-absurd-as-it-sounds-this-is-how-we-picked-up-our-abortionist-on-abortion](https://www.lifesitenews.com/blogs/as-absurd-as-it-sounds-this-is-how-we-picked-up-our-abortionist-on-abortion)

I pay the barber and I’m on my way back thinking about what I can change the next time I come in for a haircut.

This is me walking out of the barber shop.