Social Architecture 2: freedom of expression

Part of the reason of USSR failure was going against human nature. The ruling party tried to standardize everything: how people speak, what they wear, sometimes even what they think. That contradicted with human desire to create something new, to differentiate from others, to find oneself style and identity. People were jealously looking at the West, scrambling whatever bits they can get from there (jeans, VHS players, gums anyone?). This was a huge contributing factor to the crash of the empire: people wanted to be liberated.

Capitalists, on the other hand, played upon human psychology instead of fighting it: they understood the basic human needs and they pushed them above the initial meaning. Capitalists created *new* needs, and this is still how our market evolves (take any Apple product as an example). Remember what they say in business "If you are competing, you've already lost"? Now think about it again, now within the context of consumerism culture.

Now, back to USSR. The reason soviets unified everything was not only the need to control the crowd. It was also the most efficient way to manage available resources (manufacturing, materials, work force, etc). In a sense, this was a real *economy*, while capitalists were not concerned about it on the global scale. The interesting bit here that I found is that Communism in general does not require unification, it was just a technical limitation in this particular implementation.

Imagine a city of the future again (say, the Venus Project). People share everything they can (outside of their rooms). There is freedom of thought, because there of no need to control everyone: people are self-organized, and most decisions are made by a smart computer program. There is freedom of style and expression, because everyone can design their own appearance, or a commodity, and 3d-print it, thus making the design instantly available to anyone else. What happens is the advanced computing and manufacturing technologies allow us to express ourselves while still preserving the earth resources.

Concluding, it just wasn't the right time for Communism to be adopted. We needed to transit from the industrial to informational era. We needed the Internet, computing power, 3d printing, and other key technologies to be developed in order to build the new order. *Now* can be the right time. Delay half a century longer - and there will be nothing to save: our irresponsible treatment of Earth will change the atmosphere and the biosphere to the point of no return. We need to stop consuming and start thinking: about how we live, where we live, and what we can do to sustain ourselves.

Social Architecture

Human civilization has reached unbelievable power. We process petabytes of information, our eyes reach the furthest corners of the universe, while our minds dissect the finest structure of matter and energy. We are incredible species, but we really don't know how to live side by side on this planet and cooperate efficiently.

Some long time ago, our ancestors figured the concept of money. It's an artificial value used to exchange everything else with. Many of our needs can be achieved by having enough money, and we work hard to get them. Then we group into corporations - these giant money-hungry monsters with resources and power. They produce spaceship parts, they grow our food, they even teach our kids. The problem here is that they don't care about result quality, environmental concerns, and morality of their actions (think mercenaries) as long as they get the most money of it.

On the other hand, we are supposed to consume more and more for the economy to develop. Being a target of aggressive advertising and manipulation, today's adult needs a big house, 2 cars, a mobile phone, a tablet, a laptop, and 120 fashion ties in his wardrobe. If you provide these commodities to everyone, you'd realize it requires more resources than our Earth has.

In essence, money divide us. We can't efficiently work together, because money is the strongest link we have. We have organized ourselves in an unsustainable, inefficient way of life. Both present and future pose many challenges to this system:

Robotization. Modern factories are mostly self-operated. More and more human activities become outdated with the development of robotics and expert systems. Self-driving cars, for example, already on the roads, hence all the taxi/truck industry is on a timer. Eventually, humans will only be needed to operate the robots and serve each other. There will be a shortage of jobs on the market and an ever-increasing demand. Money will concentrate in the golden 1% of the society, which controls the robots and other people, thus making the rest to be slaves.

Natural resources. Our industrial revolution was made possible by the abundance of fossil fuels. We've advanced a lot, but still haven't figured out how to live in a sustainable way: fossils are limited, and the amount of energy needed to dig them goes closer to the energy it produces. We need a focused effort to develop alternative energy sources and migrate our infrastructure to be independent of the fossils. I don't see this happening at the moment, either from the governments initiatives or the corporations.

Government. We may have a vote on who is in charge, but since the government is a part of our socio-economical system, it becomes corrupted by money. Big corporations are actively (even openly) lobbying their interests, supporting candidates, and pushing their interests by any means possible. At the end of the day, the government becomes a middle-man between consumers and corporations, the goals shift, and the status quo prevails.


Beautiful Earth. Why should a corporation care about the global warming? Or that weird species of birds that can be found only in New Zealand? Or about preserving the rain forests in South America? But it's not the corporations fault, nor of its people, it's how our system works. This planet will not be able to bear with us for much longer. We pollute air, throw radioactive wastes into water, killing the living ecosystem that was established long before us.

War. War never ends. We fight for resources, for influence, sometimes just because we produce too many weapons. For some groups, war is a profitable activity, and igniting a conflict becomes their natural goal.

Health care. Imagine a cure that is accessible all over the world, costs nothing, and is able to treat all known illnesses. If such a cure is ever found, there are two possible scenarios: 1) it becomes destroyed with all the evidence, because it is not profitable; or 2) it's starts being sold to selected people only with sky-rocketed price. Health corporations are not interested in healing you quickly and efficiently (it may be a side goal, or it may be not), they are interested to suck more and more of you while you are sick. Their best interest is to charge you a ton of money, keep you for long in the hospital, and prescribe you the pills that would never help, if not make it worse.

Education. Surprisingly, the best teachers that we remember from our school days are the ones we never liked while being there. They were harsh, unforgiving, and caring only about you getting important skills and knowledge. Private education institutes are doing the opposite: they want you to like being there, they advertise heavily, and they try to convince you that the money were worth spent. Actually teaching you becomes a side effect. As a result, we get generations of worth-less over-confident individuals, who do not contribute much into our society.

Unstable economy. The flow of money does not exactly self-regulates. The free market is controlled by the laws of supply and demand. There is a lot of intermediate entities between producers and consumers. It's venture capital firms, mutual funds, stocks, global importers and local retailers, to say the least. On one hand, they naturally occur in the system of free market. On the other hand, their only job is to eventually connect consumers with producers. Naturally, they become parasiting on the system, resulting in the most powerful and rich people being them, who controls the money flow.

Better organization 

An alternative organization model is described in the Zeitgeist movies. Basically, we need to change the established value system in order to live more efficiently. We need to throw away money, strip ourselves of power, and integrate all layers of the society in a centralized manner.

Venus project is a good example. It's a self-sustainable city with no private sector, where everything belongs to the citizens. It is controlled by a computer program, which can be developed and maintained in a collaborative way. The program can efficiently distribute resources and energy, schedule manufacturing and farming throughout the whole city.

The biggest challenge is to change the motivation of people: instead of working for money, they need to work for contributing into the society, or because they simply like doing something. With todays technology, we can have most non-interesting and monotonic jobs being completely automated. Surprisingly, small communities organized in a similar manner can already be found all over the world. People are happy there, because they see apparent positive effect from their activity instead of just a pile of money as a measure of their contribution.

The change is difficult for a single person, but not so much for the whole society, because it can be achieved in an iterative manner. A group of people may unite, behaving in the "old" world of free market as a single entity. More people may join the club, build facilities, expand the farms, thus eventually unwrapping into a city. Cities may connect with each other and be sustainable without a direct intervention from the outside world (think: global warming or a world war). Of course, the ultimate benefit of this organization would only come when the whole world converts.

One can find a similarity to communism with all the possible negative associations. As an idea, communism didn't fail in XXth century. It's the implementation that failed, and this should be expected from the first rough attempt to change the social organization. The technologies weren't ready yet, and forcing people into the new system caused a lot of backslash. All in all, USSR was a world-scale experiment, which had it's moments of great glory, even though it broke eventually.

Concluding, I may have not convinced you to drop all your belongings and move into the nearest community village... But I hope my words raised your interest in the alternative socio-economical systems. I hope you'll start wondering about the laws around you that were invisible just because we are so used to them from the birth. Perhaps, you'll start seeing bad things of our world to be a little less inevitable. You may see the light of actually fixing our society at some point.

Update-1. Found this wonderful post highly correlating with my position. Must read, if only you can get through the blue background ;)

Mind shield

Travelling helps to get a clear view over the strange world we live in. Now I understand what Hollywood is, along with many other parties, as the idea hit me like a truck. And my position towards big names will never be the same again.

The ultimate goal of each corporation as well many organizations is to get your wallet. However, selling things and services directly may not be the best way to achieve it. Instead, they strike for your heart because it opens the doors to much more than just money. I'm talking about Hollywood, church, Apple, Google, governments and charity. They want you to like them, to consider them for your choice, to talk about them with your friends, to think about them at night. They want to be a part of your mind.

Humans are weird creatures. They consider themselves to be smart, but they barely understand how the environment shapes their minds. An average human does not even try to control the development of its brain, of its consciousness. And the brain just absorbs stuff chaotically, whatever happens to get in.

The story of Hollywood is simple. Long ago there was a cinema. Actors were servants, they didn't earn much, they were not recognized by random people, they were nothing at that point. And then a smart man came in and decided to change the role of the actors in the movie of life. From now on, he said, actors will be respected and well paid. He made people like them, love them and buy them. Clearly, he understood the "get into your heart" business model. As we see now, it turned out to be hugely successful. We have favourite actors, we know a lot of movies, and we pay insane amount of money to watch them at theatres. Movie business is blossoming, and every child now wants to be an actor in the future...

Let's figure out how we could protect our hearts. First off, don't watch too much of the TV. Try to evade any kind of advertising on TV, radio and on the Internet. Allocate a small part of your brain to be independent, to interrupt the flow of thoughts periodically with a simple question: "Do you really need to watch/listen/think about that?". Finally, pay more attention to stuff that really matters to you: science, art, family, philosophy, health, etc.

Suffix sorting in linear time with no extra space

Intro
Suffix array construction (or, more generally, suffix sorting) is an important task of finding the lexical order of all sub-strings of a given string. It is heavily used in data indexing and BWT compression, which I've been doing for a long time to date.
First of all, it was surprising to me to discover that the subject is possible. By linear time I mean O(N) asymptotic execution time, where N is the input array size. By no extra space I mean that the only memory required is for the input array (N bytes), suffix array (4N bytes) and some constant storage (O(1) bytes).

Induction sort (SA-IS) is the key. Authors did a great work of exploring it. The major breakthrough was to use the induction to sort LMS sub-strings, in addition to using it afterwards to recover the order of all other strings. The only thing missing was a proper memory requirement for the algorithm. Authors claim 2N is the worst-case additional memory. Let's see how we can narrow this value down.

Proof
For an input string of size N, let's define M to be the total number of LMS-type suffixes. The memory (R(N,M) machine words) required for a recursive call consists of three sections:

• Suffix storage = M
• Data storage = M
• Radix storage = K(M), where K(M) is the number of unique LMS sub-strings

R(N,M) = M + M + K(M) <= 3M/2

Now, let's split LMS sub-strings according to their length. There can be LMS of size 1. There are L2 of size 2, L3 of size 3, and so on. Let's now define M and N in terms of these numbers:
M = Sum(i){ Li }
N = Sum(i){ i * Li }

We know that LMS of different size can not be equal. Therefore, we can safely split K(M):
K(M) = Sum(i){ K(Li) } = K(L2) + K(L3) + ... + K(Li) + ...
K(Li) <= Li

We know that the maximum number of unique sub-strings of size 2 can not exceed 2^16, which is a rather small number. For the convenience let's name a=L2 and b=L3+L4+...= Sum(i>2){ Li }. It is the time to give a better upper bound to R(N,M):
M = a+b
N = 2L2 + 3L3 + 4L4 + ... <= 2L2 + 3(L2+L3+...) = 2a + 3b
R(N,M) = 2M + K(M) = 2a+2b + K(a) + K(b) <= 2a+2b + 2^16 + b = 2a+3b + 2^16 <= N+O(1)

Well, that's it. By carefully arranging the data in memory and allocating just 2^16 additional words we can perform a recursive call to SA-IS, and, therefore, construct a complete suffix array.

Code
My version of SA-IS is called A7. Advantages to Yuta's SA-IS implementation (v2.4.1):

• 5N is the worst-case memory requirement. It doesn't depend on the input.
• No memory allocation/free calls inside the sorting procedure.
• Data squeezing: using smaller data types for recursion if range allows.
• Cleaner C++ code with more comments.