Facts versus Models

Intro

Have you ever participated in an argument where countless points being raised from both sides don't change anyone's position? Each consequent attempt to convince an opponent makes you look deeper and deeper into what would make them think different from you. Eventually, you may find out that some very basic perception aspect is different. A good outcome would be one of the sides (or both) figuring out the imperfections of their model (of the subject domain) and making corrections internally, resulting in a consensus. As you see, the argument is a collision of different models, and the facts are shots fired from each side. A fact/shot can be either explained/deflected by the the receiving model (shield?), or it hits hard, exposing the model inconsistency. This, in turn, is my model of an argument process, and you are welcome to argument about it.

In the past, I've followed the facts as anchors binding our fuzzy reality to objectiveness. Whenever listening to someone, reading news, watching a show, I used to always search for the facts, extract them, and process with my models of understanding. Consequently, anything based on the believe didn't make sense to me, like religions or propaganda. Lately though, my perception focus changed from facts to models. Instead of matching the facts to my models, I started constructing images of other models and evaluating them as a whole, using facts at hand for quick verification. This is my new meta-model of perception, and I'd like to register it with this post. I'm going to analyze facts and models in general in an attempt to explain the meta-model to you.

 

Fact

Simple. One can explain a fact in a single sentence, like "it was raining for 2 hours yesterday". It is universally understandable. It's on the lower level of understanding, way below opinions and feelings, thus being totally neutral in its message.
Manipulative. A single fact is harmless, but they always come as a bunch. Supposing I showed you another planet with life forms, and all 25 of them that you've seen were white. Would that make you think all of them are white? What if I was selecting them manually just to make you think they are all white? By filtering the facts you may change the way people construct their models, and social media has been doing this for ages. One would say the filtered fact set is "incomplete", but what would qualify as complete here anyway? It's a tough question, and my answer is - the model defines completeness, being part of the reason it's so important.
You don't have to lie to steer people's minds. Next time you see (on TV, newspaper, etc.) angry people oppressed by their country leaders (with lots of natural resources), supposing you consider it a fact (see the elusive section), ask yourself questions: do these people represent the majority? would you be able to show angry people in your own country? and why they are not in the news then? etcetera.
Elusive. What is the fact anyway? Was it raining for 1h58m yesterday, or where exactly was it raining? Maybe the rain clouds moved over this time, maybe it was on and off, maybe it was just reported as raining but nothing actually happened. You may try to dig these details infinitely, and there will always be some rough corners remaining. We don't exactly interchange facts, these are just little models approximating some experience, not always valid ones. So, you can't catch a fact, it's elusive. We often agree on facts because they are being generated by a common model, common sense if you will, but it barely pretends to be more objective than any other models we exchange.
When you hear the report of a passenger plane being shot by X, ask yourself one question: does it fit into your model of X, or is it a part of transmitter's model? What is the "fact" based on: a quality video of the event, or just the words of an involved party? What could have happened that would be perceived the same way?
Useless. We can remember the fact, transmit it (a model of it, to be precise), but that's it. Actually learning from a fact involves something much more complex - a model, explained in the next section.

 

Model

Smooth and extendable. Model is what allows us to predict the future, and to fill gaps in our past. This is how the brain works - essentially, it constructs and manipulates models of the perceived world. Prediction is the sole purpose of our high-level neural system.
A historian, for example, reads the "facts" about German history, tries to construct a model in their head, including the psychological profile of Hitler, if you will. When they finished the model, checked that the facts matched (or buried the ones that did not), exchanged the idea with other historians, applied the political agenda, only then they write a book explain every little bit of what happened. The model is often hidden behind the straight facts: it's in their formation, layout, and, of course, selection (if you know a bigger subset).
Contagious. As mentioned before, we exchange models and not facts. Once you understand the model, it changes your perception of the future by proposing difficult predictions of it. Often, you'd want to adopt a model to live in a better future.
In the last century there was a couple of strong contagious models: capitalism, communism, and fascism. The first envisioned goods and freedom for everyone, the second - equality and modesty, and the third - prosperity at the cost of other nations. People fought for these ideas all over the world (WW2, Vietnam, Afghanistan, Korea, etc.), by the way constructing enormous mechanisms of spreading the idea far away (mass media, culture of journalism, generations of kids educated in a biased way, etc.). Today we read the history books written by the winners, we listen to their voice adjusted to continue zombifying us, this time only to different threats.
Complex. A model is an essence of all the facts it explains. It bears the core complexity of the domain, which can be extremely valuable. It is often complex to explain it to others, especially if it hasn't shaped up nicely yet in your own brain. These qualities make it important to always serialize the important models, which is what I'm doing here.

 

Conclusion

Pay attention to the models around you, don't focus on the facts. When thinking about a person, or arguing with one, try reconstructing how they see the world, so that it matches the actions they perform. Be ready to test your own models and be flexible to adjust them. Be skeptical about "facts" being streamed to you, try correcting them with respect to the bias of the transmitter. In a way, there are no facts, no truth, and all we got are subjective models.

Social Architecture 2: freedom of expression

Part of the reason of USSR failure was going against human nature. The ruling party tried to standardize everything: how people speak, what they wear, sometimes even what they think. That contradicted with human desire to create something new, to differentiate from others, to find oneself style and identity. People were jealously looking at the West, scrambling whatever bits they can get from there (jeans, VHS players, gums anyone?). This was a huge contributing factor to the crash of the empire: people wanted to be liberated.

Capitalists, on the other hand, played upon human psychology instead of fighting it: they understood the basic human needs and they pushed them above the initial meaning. Capitalists created *new* needs, and this is still how our market evolves (take any Apple product as an example). Remember what they say in business "If you are competing, you've already lost"? Now think about it again, now within the context of consumerism culture.

Now, back to USSR. The reason soviets unified everything was not only the need to control the crowd. It was also the most efficient way to manage available resources (manufacturing, materials, work force, etc). In a sense, this was a real *economy*, while capitalists were not concerned about it on the global scale. The interesting bit here that I found is that Communism in general does not require unification, it was just a technical limitation in this particular implementation.

Imagine a city of the future again (say, the Venus Project). People share everything they can (outside of their rooms). There is freedom of thought, because there of no need to control everyone: people are self-organized, and most decisions are made by a smart computer program. There is freedom of style and expression, because everyone can design their own appearance, or a commodity, and 3d-print it, thus making the design instantly available to anyone else. What happens is the advanced computing and manufacturing technologies allow us to express ourselves while still preserving the earth resources.

Concluding, it just wasn't the right time for Communism to be adopted. We needed to transit from the industrial to informational era. We needed the Internet, computing power, 3d printing, and other key technologies to be developed in order to build the new order. *Now* can be the right time. Delay half a century longer - and there will be nothing to save: our irresponsible treatment of Earth will change the atmosphere and the biosphere to the point of no return. We need to stop consuming and start thinking: about how we live, where we live, and what we can do to sustain ourselves.

Social Architecture

Human civilization has reached unbelievable power. We process petabytes of information, our eyes reach the furthest corners of the universe, while our minds dissect the finest structure of matter and energy. We are incredible species, but we really don't know how to live side by side on this planet and cooperate efficiently.

Some long time ago, our ancestors figured the concept of money. It's an artificial value used to exchange everything else with. Many of our needs can be achieved by having enough money, and we work hard to get them. Then we group into corporations - these giant money-hungry monsters with resources and power. They produce spaceship parts, they grow our food, they even teach our kids. The problem here is that they don't care about result quality, environmental concerns, and morality of their actions (think mercenaries) as long as they get the most money of it.

On the other hand, we are supposed to consume more and more for the economy to develop. Being a target of aggressive advertising and manipulation, today's adult needs a big house, 2 cars, a mobile phone, a tablet, a laptop, and 120 fashion ties in his wardrobe. If you provide these commodities to everyone, you'd realize it requires more resources than our Earth has.

In essence, money divide us. We can't efficiently work together, because money is the strongest link we have. We have organized ourselves in an unsustainable, inefficient way of life. Both present and future pose many challenges to this system:

Robotization. Modern factories are mostly self-operated. More and more human activities become outdated with the development of robotics and expert systems. Self-driving cars, for example, already on the roads, hence all the taxi/truck industry is on a timer. Eventually, humans will only be needed to operate the robots and serve each other. There will be a shortage of jobs on the market and an ever-increasing demand. Money will concentrate in the golden 1% of the society, which controls the robots and other people, thus making the rest to be slaves.

Natural resources. Our industrial revolution was made possible by the abundance of fossil fuels. We've advanced a lot, but still haven't figured out how to live in a sustainable way: fossils are limited, and the amount of energy needed to dig them goes closer to the energy it produces. We need a focused effort to develop alternative energy sources and migrate our infrastructure to be independent of the fossils. I don't see this happening at the moment, either from the governments initiatives or the corporations.

Government. We may have a vote on who is in charge, but since the government is a part of our socio-economical system, it becomes corrupted by money. Big corporations are actively (even openly) lobbying their interests, supporting candidates, and pushing their interests by any means possible. At the end of the day, the government becomes a middle-man between consumers and corporations, the goals shift, and the status quo prevails.


Beautiful Earth. Why should a corporation care about the global warming? Or that weird species of birds that can be found only in New Zealand? Or about preserving the rain forests in South America? But it's not the corporations fault, nor of its people, it's how our system works. This planet will not be able to bear with us for much longer. We pollute air, throw radioactive wastes into water, killing the living ecosystem that was established long before us.

War. War never ends. We fight for resources, for influence, sometimes just because we produce too many weapons. For some groups, war is a profitable activity, and igniting a conflict becomes their natural goal.

Health care. Imagine a cure that is accessible all over the world, costs nothing, and is able to treat all known illnesses. If such a cure is ever found, there are two possible scenarios: 1) it becomes destroyed with all the evidence, because it is not profitable; or 2) it's starts being sold to selected people only with sky-rocketed price. Health corporations are not interested in healing you quickly and efficiently (it may be a side goal, or it may be not), they are interested to suck more and more of you while you are sick. Their best interest is to charge you a ton of money, keep you for long in the hospital, and prescribe you the pills that would never help, if not make it worse.

Education. Surprisingly, the best teachers that we remember from our school days are the ones we never liked while being there. They were harsh, unforgiving, and caring only about you getting important skills and knowledge. Private education institutes are doing the opposite: they want you to like being there, they advertise heavily, and they try to convince you that the money were worth spent. Actually teaching you becomes a side effect. As a result, we get generations of worth-less over-confident individuals, who do not contribute much into our society.

Unstable economy. The flow of money does not exactly self-regulates. The free market is controlled by the laws of supply and demand. There is a lot of intermediate entities between producers and consumers. It's venture capital firms, mutual funds, stocks, global importers and local retailers, to say the least. On one hand, they naturally occur in the system of free market. On the other hand, their only job is to eventually connect consumers with producers. Naturally, they become parasiting on the system, resulting in the most powerful and rich people being them, who controls the money flow.

Better organization 

An alternative organization model is described in the Zeitgeist movies. Basically, we need to change the established value system in order to live more efficiently. We need to throw away money, strip ourselves of power, and integrate all layers of the society in a centralized manner.

Venus project is a good example. It's a self-sustainable city with no private sector, where everything belongs to the citizens. It is controlled by a computer program, which can be developed and maintained in a collaborative way. The program can efficiently distribute resources and energy, schedule manufacturing and farming throughout the whole city.

The biggest challenge is to change the motivation of people: instead of working for money, they need to work for contributing into the society, or because they simply like doing something. With todays technology, we can have most non-interesting and monotonic jobs being completely automated. Surprisingly, small communities organized in a similar manner can already be found all over the world. People are happy there, because they see apparent positive effect from their activity instead of just a pile of money as a measure of their contribution.

The change is difficult for a single person, but not so much for the whole society, because it can be achieved in an iterative manner. A group of people may unite, behaving in the "old" world of free market as a single entity. More people may join the club, build facilities, expand the farms, thus eventually unwrapping into a city. Cities may connect with each other and be sustainable without a direct intervention from the outside world (think: global warming or a world war). Of course, the ultimate benefit of this organization would only come when the whole world converts.

One can find a similarity to communism with all the possible negative associations. As an idea, communism didn't fail in XXth century. It's the implementation that failed, and this should be expected from the first rough attempt to change the social organization. The technologies weren't ready yet, and forcing people into the new system caused a lot of backslash. All in all, USSR was a world-scale experiment, which had it's moments of great glory, even though it broke eventually.

Concluding, I may have not convinced you to drop all your belongings and move into the nearest community village... But I hope my words raised your interest in the alternative socio-economical systems. I hope you'll start wondering about the laws around you that were invisible just because we are so used to them from the birth. Perhaps, you'll start seeing bad things of our world to be a little less inevitable. You may see the light of actually fixing our society at some point.

Update-1. Found this wonderful post highly correlating with my position. Must read, if only you can get through the blue background ;)

Circles of the Home Automation hell

Introduction

Just a week ago I knew nothing about IoT. All the buzz was meant (I thought) for someone else, but that changed when I bought a house. Suddenly, a strange desire was born in my mind: to connect entities (doors, rooms, speakers, etc) inside my house, to monitor and control them as a system. Thus, I started looking into Home Automation, and with each new iteration I was diving deeper and deeper into the hell of technological diversity.

 

0. Complete Solutions

There is a lot of companies (like this one nearby) offering to install their proprietary home automation system for a big sum of money on the case by case basis. That wasn't acceptable for me, because I wanted to know what my options were, and expand gradually.

 

1. Isolated subsystems

Some companies specialize only in a specific domain. For example, Lutron is known for its lights and dimmers, which come with remotes. Phillips Hue makes the best LED lights, also remotely controlled. Nest is a decent (and awesome looking!) learning thermostat. Going wider, Vera allows you to buy components that you need and attach them to the system dynamically. Naturally, I started asking myself why these things can't be organized together, what language to they talk to each other?

 

2. Wireless Protocols

It turned out, there is a lot of diversity in wireless means of communications. You thought that WiFi and Bluetooth are enough for everyone? Welcome to the real hell:

ZigBee - perhaps, the oldest protocol (outside of X10, which we'll skip). It's looking decent on paper: support for mesh networking, low power consumption, huge group of supporters. It is ISO certified, and you can find numerous open source libraries and protocol implementations. However, the devil hid in the details, again. There are two incompatible kinds of hardware: Series 1 and Series 2. There are different profiles of communication, and you can't easily mix and match those. Finally, I tried to look for something specific, like a temperature/humidity sensor, and it came short of options for the real "Buy" button to appear. It appears that Zigbee has some fundamental issues (maybe those I mentioned) that pushed the alternative developments.

Z-wave - the most available (in terms of hardware) protocol, which also claims to be very smart in design (mesh networking and such). It is really straightforward to find actual devices, but unfortunately difficult to program them: the API is opened only after buying the SDK and signing an NDA. Open-source implementations exist but seem to be rough and incomplete.

EnOcean - the european-origin protocol with the main focus on self-powered devices. There is not much out there to read about it, but the limited range of compatible devices is available from a single manufacturer. The promise of battery-less components seemed very appealing, but it got compensated by the price of those.

Other things I didn't research include Insteon, Bluetooth Smart, WiFi (for IoT), ClearConnect (used by Lutron), and Thread (used by Nest). The last one seems exceptionally promising, but no public information is available yet about it.

At this point, I realized that if I'm gonna control my devices, they had to use the same protocol. Alternatively, I found a market of cross-protocol universal hubs, such as Wink, Revolv, Staples Connect, and SmartThings. While they do allow using devices from different networks, I found only Wink providing an actual API, which I doubt works flawlessly. If you don't intend to be in full control, this may be your last stop (for good).

Another problem, adherent to most solutions, is the requirement of Internet access. While I do appreciate a mobile app to access my home system, I believe it should not involve a cloud server. The cloud is a privacy hole and yet another point of potential failure. My system should be as much self-contained as possible.

Thus, I wanted to go deeper... The hub, I imagined, could be a headless Raspberry Pi with a RF module (like XBee) that I'd program myself in Rust. Fortunately, Pi supports modules for all major RF protocols. The hub would host a website for online access and schedule my house in a 24/7 mode. The only problem was locking into a single protocol (and its API), because I didn't want to hook up and dig into multiple protocols at once.

 

3. Micro-controllers

I noticed that sensors without RF modules are very abundant and cheap. What if I connect them to my own network physically, by having a micro-controller and a RF module attached by hands? That perspective made me look into simple computing cores:

Arduino - the most known family of boards, featuring dominantly the AVR family of controllers. Sadly, LLVM (and hence Rust) does not support this platform as a target, and the cheapest ARM core (Arduino Due) is more expensive than Pi. If not for these factors, I'd go with Arduino in a heart beat, for it's rich documentation and wide community support.

STM32 family - the most open-source friendly ARM chips out there. Rust has a Zinc project of running on bare metal, that was developed for this chip. Prices start as low as 8$, and there are extension boards available.

Tiva C launchpad kit from Texas Instruments - the most impressive ARMs in terms of website navigation and availability of standard extensions. Bare boards start at 13$ and seem very solid, thus being my current choice.

Freescale Freedom Boards - the most diverse family of ARM chips. I found it difficult to navigate their website and to figure out what exactly I need there. Can't see any extension boards in particular, but I'm sure there are some. Prices start at about 16$, though these boards include some LEDs and switches for demonstration purposes by default.

 

Conclusion

As a starting point, I decided to order one of the low-end ARM chips, and try to get anything programmed to it in Rust. That will take me a while... I may end up contributing some code into Zinc, or even shifting to robotics after-wise, because that's where MCUs really shine. At the same time, I'm going to buy a Nest thermostat with a couple of Nest CO detectors for my family to enjoy while I'm tinkering with low-level stuff. If I ever reach the point where I could control my HVAC and other things remotely by my own program, I'll be happy to replace Nest with something dummier, or just hack it to get the root access to HW.

If thought about home automation, have some solutions installed, or are building your own hub in the garage - please share, as I'll be happy to learn from your experience. If you didn't care about IoT and my article made you interested - you are welcome ;)

Vision

Vision. I often heard this term, read about it, though I knew what this is. I thought, when there is a person in charge, who is a highly skilled engineer, he can see what others can not just because of his engineering capabilities and the fact he goes thinking deeply about the problems (not only existing ones, he is also in a constant search for the new ones). Apparently, it's not that simple.

It turned out, vision is not a consequence of engineering skills but something very different. It's an extraordinary ability to see the future, a hypothetical future of a product evolution. By seeing this, a revisionary can drive product development in leaps, thus making it a revolutionary progression.

It is important to realize this is not "just" about development time (or, the availability of shortcuts). Evaluation is all around us. We evaluate everything we are doing in order to learn from it and adapt. Others evaluate our work in order to figure out if they want to invest in it. Thus, being able to leap forward gives you an instant advantage in terms of evaluation outcome, which transforms the benefit of having a vision from quantity (of time) to quality.

I faced the vision problem when tried to design a large system, in collaboration within a team. I realized (for the first time) that it's not the engineering skills that we lack, but rather a clear vision over where we are going. It doesn't even matter if you are an architect, lead developer, or the god himself. If you got the vision, you'll be heard.

Aside from being mysterious, vision is still a human skill. I wonder if it can be trained, like any other human skill we have. What kind of activity would that be? I don't believe that by just thinking and brainstorming we get any better in visioning in general, we merely dig harder at a specific problem instead. There must be something more generic. Perhaps, playing the music?..

3C Rules of Personal Development

Create
You know, do stuff. Experiment, hack, write programs, build houses. Creating means going against nature in some way: second law of thermodynamics rules out everything to go into chaos, while creation is about organizing matter or information. Creation is easy and natural behavior of children with their rich imagination, but keeping up with it when you get older requires dedication.

Collaborate
Working on something in isolation can be reasonable, but the potential of collaboration is greater. Discuss your ideas with friends, relatives, and even with random people on the Internet. Visit conferences to hear other peoples ideas, form work groups, and adjust your own goals. A well done argument may multiply each individual intellegence with regards to solving the target problem, when the idea is being ping-ponged between brains, evolving with each hit.

Complete
Creation process is an engine, and to keep working it needs to complete cycles. Finishing stuff gives you fuel to move to the next idea, and allows to draw right conclusions by analyzing the full cycle. Publish your work on the web, give other people a functioning product, not just a bunch of scrap and a github repo link. Receive recognition, push your goal bar higher, and move on.

Quest for the best scene format

A graphics engine needs to know how to compose a scene from a given set of resources, such as: meshes, skeletons, and textures. This is what scene file is for - it's a document that describes relationship between basic resources, joining them into a system that can be effectively processed by the code. This file is composed either by hand (if the scene is small), or by the exporter script from a 3D modelling program. During the evolution of KRI engine the scene format changed several times. I'll try to review the development history and analyze various formats I used, based on the personal experience.

0. Composed in code: kri-1, kri-2

This is where we all start: just slapping entities on the screen directly.
Lang:    C++
Pros:
    -no export stage
    -no need to validate
    -no parsing
Cons:
    -non extensible

1. Custom binary: kri-3

All scene data and resources were stored in a single binary file of a custom format.
Lang:    Boo
Pros:
    -no external libs
    -fast parsing
Cons:
    -non human-readable -> difficult to debug
    -difficult to validate
    -resources are not separate

2. XML: kri-web

XML is a well-known document format, it has the greatest language/tool support. Besides, that's what we used at my former employer company.
Lang:    Dart, XML
Pros:
    -built-in validation with Schema
    -support for default values
Cons:
    -need to keep Schema synchronized with exporter/loader
    -too verbose
    -bloated loading code (no 1:1 data representation)
    -not clear what to put into attributes -> design ambiguity

3. JSON: claymore-engine

This is where I discovered JSON, and it immediately appealed to me because of the simple syntax and its 1:1 reflection with the data. Fortunately, this is the only format Rust had a built-in support for. However, it turned out to be a poor choice for the scene description due to the lack of heterogeneous structures.
Lang:    Rust, JSON
Pros:
Cons:
    -no heterogeneous structures -> difficult to read

From there I started looking for something like JSON but to describe the document instead of the data. I looked into YAML, which seemed nice, a bit more complex, and not supported by Rust. Then I found Candle Object Notation, which seemed like a non-ambiguous and compact version of XML, plus the 1:1 mapping to data. However, the format is not that well documented and supported... "It would be nice to have the same object initialization syntax as Rust" - I thought when this idea hit me like a train: why not use Rust then?

4. Rust: k-engine

Let's just export a rust source file, which will be compiled with the application.
Lang:    Rust
Pros:
    -free validation (your code is the spec)
    -no run-time parsing -> instant loading, no run-time errors
    -no need to learn new syntax
    -compact (no external file needed to run)
Cons:
    -need to compile the scene
    -bound to the language

This approach seems to be the perfect solution for my scene format. The only thing that worries me is that it depends on Rust compile times. Though, we can still parse Rust code at run time, if we want, while still verifying it at compile time. You can see an actual export result here. It is compact, easy to read, and elegant.

My Internet

I've been accustomed to the Internet as we know it: fire up a browser, read email, feeds, visit FB/G+/Twitter, buy something on Amazon with a Visa card. It is indeed convenient, especially if you don't care to look under the hood, or explore the limits of your freedom. And the fact is: there are big companies there (providers of "free" services) that gather all information about you.

One of the ways to use that information is to chose the advertising that you'll see. Honestly, I don't care about ads too much. Most of the time I block them anyway, and when I see them - I'll appreciate a featured anime figure more than some silly pills. But ads are just the tip of the iceberg, the only part of it we actually see. The real problem is the power you give them, the power that limits your potential, because no one cares about your weird habits until you become big.

In an ideal society, everyone can know everything about everyone. But we, as a species, are not ideal, thus making it a matter of protection to choose what information to share, and what to hide. Once your information can be bought, you never know who and when may turn it against you. Imagine a robber aware of your vacation schedule. The security question stands right near the privacy one. Your information is stored in a centralized manner: it could be denied of service, or it could be stolen - it's vulnerabile.

Now, how do we work around that, while still keeping it simple and convenient? There are several solutions for different sub-issues:

DNS. Generally provided by your ISP, thus may have some areas blocked (i.e. Wikileaks). They know wherever you go, and also redirect the "address not found" queries.
Solution: OpenNIC, any neutral DNS like Google DNS

File sharing. Exchanging music, books, and movies is prosecuted by RIAA & MPAA, even if you give it to your friend and delete it locally. They want you to rent things for an undefined period instead of owning them.
Solution: Torrent, private hosting if you can afford it.

Social network. This is where you expose the most of yourself. You need to preserve the rights to the content you create, and to share it only with those you care.
Solution: Disapora*.

Money transfer. Your Visa/AMEX/Mastercard knows everything you buy, everywhere you travel, and steals around 3% for each transaction. Also, you never know when your government decides to print more money, making whatever you have less valuable instantly.
Solution: Bitcoin.

Those solutions will only become valid once they gain a critical mass of users. I hope that my post aids this goal a little, making Internet a better place in the nearest future.

What I know about Computer Graphics

I've been working closely with CG both professionally and as a hobby for the past 5-6 years. I've been making games and developing engine architectures. Latest developments can be tracked on Claymore Dev blog. I've seen different techniques, tried many others, even written articles about them in big books (GpuPro-3, OpenGL Insights). And the funny point is: I still don't know how to build engines... All I know is how bunch of known techniques may help or screw you up, based on personal experience.

Uber-shaders
Problems: Difficult to maintain due to bad granularity and monolithic approach. Unable to extend from the game side.
Alternative: Shader compositing. In OpenGL you can extend the functionality by either linking with different shader objects, swapping the subroutines, or by directly modifying the source code.

Deferred shading
Problems: Very limited BRDF support. High fill-rate and GPU memory bandwidth load. Difficult to properly support MSAA.
Alternative: Tiled lighting. You can work around the DX11 hardware requirements by separating lights into layers (to be described).

Matrices
Problems: Difficult to decompose into position/rotation/scale. Take at least 3 vectors to pass to GPU. Obligation to support non-uniform scale (e.g. no longer skipping invert-transpose on a 3x3 matrix to get a normal matrix).
Alternative: Quaternions and dual-quaternions. Both take 2 vectors to pass.

Context states
Problems: Bug-hunting is difficult because of bad problem locality. Assumptions over the context are easy to make, but if you decide to check it with assertions, why not just pass the whole state instead?
Alternative: Provide the whole state with each draw call. Let the caching work for you.

C++
Problems: Memory management and safety. Compiler-generated copy operators/constructors. Pain dealing with headers and optimizing the compile time. Many many lines of code.
Alternative: Rust. Other "safe" languages (.Net family, Java, Python) are not as low-level and often trade performance for safety (i.e. global GC phase causes an unacceptable frame rate interruption).

All I actually know is that there is a thousand and one difficult architectural issues of the graphics engine, and there is no silver bullet to most of them. For the most common solutions I listed possible alternatives, but they are no near being flawless. I hope that one day the amount of experience I get will magically transfer into the quality of my decisions, and I will finally know the right answers.

Rust

Today's early morning I woke up with a single thought reflecting loudly in my brain: "Dart was a mistake, it was not made for me. I should look for some statically typed practical language instead". Even though my KriWeb project (written in Dart) was not actively developed, I agreed (with my dreaming counterpart) that the instrument I choose for this project iteration is far from perfect. Suddenly, I felt the urge to look for something ideal, something that seemed so real as if I was reading its specification the other day... And I just needed to recall its name...

I started looking for it on the web. There were many interesting suspects among new languages. Ceylon, for example, features immutability by default (which highly encourages functional style), which seemed very familiar and close to what I looked for. It is a very nice language all in all, but it's currently running on Java VM, and was heavily inspired by it, what pushed me off. Go sounded attractive due to the strong support from Google, however disappointed me by its lack of user generics. Zimbu looked too original, while Haxe seemed to pretend covering too much use-cases. I've reached  the 5-th page in google search results, and there still wasn't any trace of it. Maybe it was a dream?..

One step away from stopping my search, I stumbled upon this Holy Grail of programming. Name is Rust, developed by Mozilla Foundation. Suddenly, I remembered this shining website interface, this clear language specification that I read a while ago. I found it, at last! Let me explain why I was so happy:

• Strong static typing with inference, only explicit mutability. This is so right and so rare to see at the same time. Unlike Dart, most of my mistakes will be found at compile time.
• No page faults while still compiling to native code. Memory model is protected and guaranteed to work without access violations under normal circumstances. It has a potential for C-like performance, hence being a better tool for various tasks.
• User generics with constraints, pattern matching (Haskell-style). Yes, it took the best from my beloved purely functional language.
• Less statements but more expressions and closures. This makes it even more sleek and functional.
• Syntax extensions. Hello, Boo macros!
• Structure compatibility with C. Using external API's (i.e. OpenGL) gets easier.

Overall, the language and its environment seem very nice. It is simple yet powerful, and feels very promising. I'm looking forward to work closely with this gem, and I'm very excited :)

Mind shield

Travelling helps to get a clear view over the strange world we live in. Now I understand what Hollywood is, along with many other parties, as the idea hit me like a truck. And my position towards big names will never be the same again.

The ultimate goal of each corporation as well many organizations is to get your wallet. However, selling things and services directly may not be the best way to achieve it. Instead, they strike for your heart because it opens the doors to much more than just money. I'm talking about Hollywood, church, Apple, Google, governments and charity. They want you to like them, to consider them for your choice, to talk about them with your friends, to think about them at night. They want to be a part of your mind.

Humans are weird creatures. They consider themselves to be smart, but they barely understand how the environment shapes their minds. An average human does not even try to control the development of its brain, of its consciousness. And the brain just absorbs stuff chaotically, whatever happens to get in.

The story of Hollywood is simple. Long ago there was a cinema. Actors were servants, they didn't earn much, they were not recognized by random people, they were nothing at that point. And then a smart man came in and decided to change the role of the actors in the movie of life. From now on, he said, actors will be respected and well paid. He made people like them, love them and buy them. Clearly, he understood the "get into your heart" business model. As we see now, it turned out to be hugely successful. We have favourite actors, we know a lot of movies, and we pay insane amount of money to watch them at theatres. Movie business is blossoming, and every child now wants to be an actor in the future...

Let's figure out how we could protect our hearts. First off, don't watch too much of the TV. Try to evade any kind of advertising on TV, radio and on the Internet. Allocate a small part of your brain to be independent, to interrupt the flow of thoughts periodically with a simple question: "Do you really need to watch/listen/think about that?". Finally, pay more attention to stuff that really matters to you: science, art, family, philosophy, health, etc.

First DirectX impressions

During the last half a year I had a great opportunity to work closely with DirectX on a production scale. From the very beginning, I had a suspicion that the technology is a big joke. It didn't start with little things, no, it started with a full-scale attack on my OpenGL-friendly brain. Let me name the offenders:

Render state. You are not in a full control of it. DX runtime may change it without your concern. In particular, this happens under DX10+ when you bind a texture resource that is also one of the render targets. Debug runtime will notify you in the log but the regular one will just do it silently. And then you wonder, where did the texture go? Unsurprisingly, PIX, the hammer of DX frame profiling, is not able to handle this behavior correctly, so debugging one of these little bugs may cost you a really long headache. In DX11 we can see a new flag that allows binding a depth-stencil texture as read-only, allowing to sample from it. As a result, you have to split the rendering paths: copy the depth for DX10, and use the flag for DX11.
In contrast, OpenGL gives you an undefined behaviour whenever you want to read and write at the same time. While it seems suspicious at first, in practice you don't sample from the texture being rendered to, so your program works as expected, and no state is corrupted. Moreover, your GL program doesn't need a new flag, or a depth texture duplicate: all you need is to disable depth/stencil writes, and you can read them. Concluding, while DX creates and fixes the problems of its own with new versions, OpenGL just works as expected.

Documentation. The official source of knowledge about DX is MSDN. Unfortunately, you are not able to track all changes that go there. I don't see any revision history. My co-worker was following the CHM documentation bundled with our DirectX SDK, and according to it, CopyResource() can not be used if one of the surfaces is multi-sampled. He ended up copying a surface using a full-screen quad with a designated shader... And only then I discovered that the online version is different: for DX 10.1 the function actually copies multi-sampled surfaces too.

Another example is D3D11_RASTERIZER_DESC structure. There is a MultisampleEnable member, which (surprise!) affects only line rendering under DX10+, while affecting all MSAA rendering under DX10 and below. Yes, I know there is also AntialiasedLineEnable flag, but how does this make it any less confusing?

The situation gets worse as you dig deeper. As an example, there is a texture object in HLSL. According to MSDN, you have to explicitly specify the number of samples in the template. Though, I'm not sure, maybe the page is fixed while I'm typing this. Anyway, in practice, under DX10.1+ you can skip it. That's where you end up scavenging all little details of these presentations, scanning the forums, and trying to guess logically. The DX knowledge is like a secret cave with treasures, where some companies (Epic,Crytek) know them better then others.

OpenGL, on the other hand, provides a strictly versioned document. You can download any version of it, see the changes highlighted, and find everything you are looking for. You don't need to scavenge the forums: if something works differently from the specification, it's most likely a bug, and not a feature.

Sloppiness. You can do many things incorrectly, and DX runtime will still try its best to let your application work. For example, you can sample from a multi-sampled texture bound as a regular one. DX will automatically resolve the pixel before sampling, if it's possible. Or you can assign a float3 to a float in HLSL, and it will still work (can probably be fixed by a strict flag in HLSL compiler). Such robust behaviour is very welcome on the end-user side, but developers need to be sure the code is valid. I would prefer it to crash hard on the first error encountered, or at least return some error code (hello, OpenGL). I understand that, again, one can use the debug runtime, see the error log, and figure this out. But the truth is, most development goes with a regular runtime, because the debug one is damn slow. And even on that you would have to trace through the suspicious instruction to see the new error in the log - that's not how exceptions should be handled.

All in all, DirectX makes an impression of being made by amateurs, who got the power to talk to hardware developers. It's not developer friendly, it's not blazing fast, it's not something to compete with OpenGL. I admit that's a bit of an emotional over-statement, and one could expect something like that from me. I will continue learning DX technology, and I hope to discover some real gems there, if they exist at all.

KriWeb project future

Introduction

KriWeb is my hobby 3D engine, the 4-th incarnation of KRI technology. I've been working on it for the last half a year in my spare time. Recently, I finished implementing the heart of the concept - shader composing pipeline.

Technology

In short, shader compositor was designed to decouple rendering technique code from the material and mesh modifiers. The material provides a set of functions to the pixel shader, which are used by the technique shader code. The technique knows how to apply an arbitrary stack of geometry modifiers (e.g. skeletal animation, morphing, displacement). without knowing anything about the actual modifiers used by the entity. The shader compositor assembles all these parts together in a linked shader program, that is associated with the entity.

As an example, we can imagine a material that provides a pure BRDF function. A technique knows about scene lights, and uses this BRDF to evaluate lights contribution to a surface point. An underlying mesh gets modified by, to say, a skeletal animation. These pieces of functionality will be glued together automatically to display a shiny animated object for you. While the Demo already shows it working, a better one could be made to harness the full power of shader compositing.

Future

Now it is time for me to evaluate the path I made, and to figure out the vector of progression for the nearest future. With all KRI incarnations (as with most of existing hobby engines), there was always a big issue chasing me - the lack of application. I was dodging it as I could, but in the end the engine dies without an application. I don't want to see KriWeb old and weak after several years of development. If it is to die, let it die young, and remember its technology shining brighter than the sun.

In other words, I don't want to continue the development of KriWeb until the real application is found. It may be either my own new project, or a cooperation with someone, but it has to be something good. It's not like I have a lot of free time now - working at Rockstar is pretty close to a dream job, and my skills are needed there in full while making the next big thing. Cheers!

Revolution in Game Development

Hardcore gamers have been struggling to see good games in the past 10 years. With each new release, each new title, or a demo, I've been looking with hope that it can be something incredible. But, generally, there wasn't a single great game, just a couple of good ones instead. Today, all big titles are targeted at soft-core audience, because it's easier to make and sells good. Fortunately, this is going to change in 2013, and the roots of the revolution are visible now.


2013 will be the beginning of the next golden age of gaming. The epicentre of the last one was around 1998, and I'm sure there was at least one more before it in 80th. The reason for games to change is the revolution in relationship between the developer, users, and the publisher on the way to extinction. The key concepts of the new era are digital distribution and crowd funding. This revolution is happening today, and the leaders have already shown up:

1. Steam (2002): the flagman of digital game distribution built by Valve. Steam helps PC developers to sell the game, and advertise it, without having a publisher. Steam has also shown us that games don't need to be so expensive, and the price can go down faster after the release, especially if the game turned out to be not as good as advertised.

2. Humble Bundle (2010): demonstrated the effectiveness of pay-what-you-want business model applied to indie games. Plus the fact that copyright protection is undesired: both Humble Bundle and GOG service provide only DRM-free content. An interesting discovery was that Linux/Unix users are ready to pay more than Windows gamers.

3. Minecraft (2009): an original game that became popular in the open alpha state. Minecraft was not the first, but it was the brightest and incredibly successful example of the game sponsored by the live user community. People realized that they can not only pay for existing games, but also influence the future by investing in the ideas they like.

4. KickStarter (2008): a portal that connects game developers with gamers, who are ready to invest their money. Millions of dollars are gathered around ambitious projects, exceeding developer expectations by a large factor. It is the final link in a chain that leaves no place for big fat publishers. Well, except for console games... for now.

I'm calling everyone to sponsor the games you would really enjoy! This revolution will make 2013 a wonderful year of games, which would be able to compete with veterans of 1998. For a complete picture, here are the games I'm proud to support:

Wasteland 2
ShadowRun
Age of Decadence
StarFarer
Tortured Hearts

A Perfect Game

Computer games are substantial part of my life. They bring new ideas, unique experience, and challenge my tactics and reflexes. I always think about qualities of game in general, and try to judge existing games upon these characteristics. There is a game that reached my heart, and I would like to tell you about it.
I played all genres, with RPG being the favorite. I adore Fallout 1/2, Arcanum, X-Com 1/2, Unreal-1, Planescape: Tourment, Baldur's Gate, Jagged Alliance, MechCommander, and other classics of 90th. Since that golden age the overall depth of the content was quickly decreasing, while the appearance was getting more and more detail. These titles are well respected by a limited community, but the subject is not among them. The subject came out shadowed by the titans, and having unique language, enormous system requirements, and little-to-no advertising, was left with no chance to shine.

The game Vangers was created by pure geniuses from russian K-D Lab studio. It features unique voxel-based terrain engine, novel-level futuristic story, and a gameplay mixed between RPG/action/simulation. The world you are literally thrown in lives by it's own laws (unlike XCom/JA/MC, where everything is user-centric). You are not special there (unlike Fallout/Arcanum/Planescape/BG), in fact there is a thousand of others, who are faster, stronger, and even able to reach the story goals before you. The world behaves as a living organism: try to leave the control for a second, and you will notice swarms of little creatures flying, swimming, crawling in the terrain; other vangers rushing in a race competition; global world cycle changing from winter to warm summer... The landscape is dynamically persistent through the game: once destroyed a bridge in a crazy fight - and you have to look for another way to cross the river. The role-playing is based on your very actions, not on some digits in your stats. On the one hand, you are absolutely free to do anything there, even the suicide can be your game ending. On the other hand, there is a strong story line that keeps you fully motivated to explore.


The game didn't get proper reception. Some people love it, some hate it, others don't understand, or simply never heard of this gem. It could make a perfect MMOG, but it already has various multi-player modes, and the games are still hosted. I enjoyed *playing* the story, because this is the game play in its perfect sense. From that moment, I started looking at the real world with eyes of a vanger.

Report: the end of 3rd development age

Year 2012 started with a new vision over technologies I want to use in order to achieve the same old goals. The 3rd age of my personal projects lasted for 2.5 years and is over now. I'd like to make a small overview over what is going to left in the past, and what are my new friends in the nearest future.

Boo, the dinosaur of the old age.
The major fault of Boo for me was its immaturity. Imagine developing a killer feature in your project and then getting "unknown compiler exception" after you changed tons of code without being able to compile. Next thing you do is spending a day in narrowing down the issue, providing the test case for the bug, and then the next day trying to work around it, "temporarily". That's the time you could spent doing something important for you, not for the language creator and community. I tried to patch the imperfections of generics implementation with smart AST macroses, and it was indeed fun.

The other faults come from the Boo main platform - .Net/Mono. I've met some serious inconsistencies and ambiguities there. For example, OOP polymorphism can be achieved in two ways: using virtual methods or via implicit interface implementations. I had to use both, because this is how platform is set up.

Portability was the last major issue. While you could safely copy binaries on Linux/MacOS and try to execute them, this didn't work smooth in practice, not to mention that tablets/phones where completely out of scope. Errors about some library of some version not being found on a target platform drived me crazy.

I still like it and will continue to use it in a Unity-based project. Working with Boo was an important stage in my professional growth, but it's time to move forward now.


Dart + WebGL, the ultimate portability solution.
OpenGL is developing too quickly for me. It's difficult to constantly adapt to new features and redesign the system. WebGL is much more stable. It can potentially work on any platform, without any platform-specific client code. Back in C times I could provide sources for you to build them locally and execute. In Net/Mono/Java times I could provide binaries that are likely to just work. In Web times I just give you the link...

The 4th iteration of KRI engine will be developed from scratch to work on WebGL. Everyone would be able to instantly see the result of my efforts without setting up any development environment. I've got much more experience now, worked with 3 different mature engines. I have a clear understanding of the goals and principles upon which the new engine structure should be built. Besides, the philosophy of not doing anything heavy on CPU and generating as much as possible fits WebGL very nicely.

Dart comes as a perfect replacement for JavaScript here. Cleaner code, better OOP and FP integration, and finally the opportunity to be in the first wave of new WebGL applications. I can't imagine doing anything serious with JS, and I like Dart very much so far.


Haskell, the global paradigm shift.
Meeting Functional Programming changed my mind, and I'll never be the same again. No, I'm not going to stop using imperative languages, but the way I look at the code is very different today. Haskell experience helped me to develop a vision of really clean and error-less code. For each piece of logic now I prefer to know exactly the input data and the result, removing all implicit and hidden flows. For example, any singleton or a static piece of data I consider dangerous.

Haskell is now the language of my AI experiments. While still learning, now I think on a higher level of abstraction, which simplifies the development and allows to concentrate on ideas more than on tools.


Conclusion.
Learning new principles is very beneficial. It's not just about new abilities, it's also about looking at old things under very different perspective. I've had a lot of fun with OpenGL-3 and Boo, they were a real step forward comparing to the my C/C++ only 2nd age. But even more fun waits ahead. New development principles, that I'll cary through the 4th age, are very promising, and I'll do my best to realize their potential in full.

Functional thinking

I started reading a book "Learn You a Haskell". It is truly wonderful, explaining complex things in a simple friendly way. Haskell itself seems to be the functional language. It's pureness makes you feel writing math  lemmas and theorems in a formal language, it is a very different feeling from C++/Boo I used to program in.

Surprisingly enough, functional programming makes us care about the goal, or a shape of the result. You answer questions like "What should it look like? What does it consist of?". At the same time, imperative programming involves asking the question "How?" most of the time.

As a first milestone and a practical task in learning Haskell I wrote the Burrows-Wheeler Transformation (BWT). I never thought it could be implemented in just 10 lines (not counting qsort), and remain nicely readable after that:

qsort :: (Ord a) => [a] -> [a]
qsort [] = []
qsort (x:xs) = qsort left ++ [x] ++ qsort right
where left = filter (<=x) xs
right = filter (>x) xs
bwt :: String -> (Int,String)
bwt input =
let buildMx [] _ = []
buildMx (x:xs) ch = (x:xs,ch) : buildMx xs x
mx = buildMx input (last input)
sorted = qsort mx
output = map snd sorted
base = head [ i | (i,(s,_))<-zip [0..] sorted, s==input ]
in (base,output)

Suffix sorting in linear time with no extra space

Intro
Suffix array construction (or, more generally, suffix sorting) is an important task of finding the lexical order of all sub-strings of a given string. It is heavily used in data indexing and BWT compression, which I've been doing for a long time to date.
First of all, it was surprising to me to discover that the subject is possible. By linear time I mean O(N) asymptotic execution time, where N is the input array size. By no extra space I mean that the only memory required is for the input array (N bytes), suffix array (4N bytes) and some constant storage (O(1) bytes).

Induction sort (SA-IS) is the key. Authors did a great work of exploring it. The major breakthrough was to use the induction to sort LMS sub-strings, in addition to using it afterwards to recover the order of all other strings. The only thing missing was a proper memory requirement for the algorithm. Authors claim 2N is the worst-case additional memory. Let's see how we can narrow this value down.

Proof
For an input string of size N, let's define M to be the total number of LMS-type suffixes. The memory (R(N,M) machine words) required for a recursive call consists of three sections:

• Suffix storage = M
• Data storage = M
• Radix storage = K(M), where K(M) is the number of unique LMS sub-strings

R(N,M) = M + M + K(M) <= 3M/2

Now, let's split LMS sub-strings according to their length. There can be LMS of size 1. There are L2 of size 2, L3 of size 3, and so on. Let's now define M and N in terms of these numbers:
M = Sum(i){ Li }
N = Sum(i){ i * Li }

We know that LMS of different size can not be equal. Therefore, we can safely split K(M):
K(M) = Sum(i){ K(Li) } = K(L2) + K(L3) + ... + K(Li) + ...
K(Li) <= Li

We know that the maximum number of unique sub-strings of size 2 can not exceed 2^16, which is a rather small number. For the convenience let's name a=L2 and b=L3+L4+...= Sum(i>2){ Li }. It is the time to give a better upper bound to R(N,M):
M = a+b
N = 2L2 + 3L3 + 4L4 + ... <= 2L2 + 3(L2+L3+...) = 2a + 3b
R(N,M) = 2M + K(M) = 2a+2b + K(a) + K(b) <= 2a+2b + 2^16 + b = 2a+3b + 2^16 <= N+O(1)

Well, that's it. By carefully arranging the data in memory and allocating just 2^16 additional words we can perform a recursive call to SA-IS, and, therefore, construct a complete suffix array.

Code
My version of SA-IS is called A7. Advantages to Yuta's SA-IS implementation (v2.4.1):

• 5N is the worst-case memory requirement. It doesn't depend on the input.
• No memory allocation/free calls inside the sorting procedure.
• Data squeezing: using smaller data types for recursion if range allows.
• Cleaner C++ code with more comments.

AI concepts mind dump

I've been thinking about AI structure for at least 7 years already. Thinking a bit and not too often, but steadily advancing. There is an open-source project where I'm doing all the practice, but today I would just like to dump the theory, giving my mind the more free space to advance.

Agent - a subject of AI, has a mind in a form of a neural network (NN).

In a time-step-based approximation an agent takes input from its receptors. The input is going through the NN producing some charge on the actor neurons. One of the actors is chosen each time step to be performed. The result of the action is spread back through NN, adjusting and optimizing the network structure.

Neuron - a base element of the NN, has:

• A list of pointers to output neurons (axons).
• Current charge and response.
• A memory map of stamp->charge.

Weights.

In a classical NN model inputs have different weights. This can be achieved in a uniform weight model (all inputs/outputs of a neuron are equal) if you start adding similar neurons with inputs being a subset of the original neuron inputs. Therefore, I'm going to throw weights off and take each input/output equally strong, distributing the signal uniformly.

Adaptation.

One of the experiments showed me that you can not go on with a static NN structure. You have to add, remove links and other neurons at least logically (one might say that biologically nothing new is created during the thinking  process, but I don't have any proof for that and I don't need it). Therefore, there is no point of adjusting the input weights by the agent adaptation process - pure neuron/link creation/removing heuristic would produce the same result and is required anyway.

Signal.

A neuron is not a function - it's just a multiplexer and a transmitter of the signal (which can me mathematically expressed as a function). Therefore, it has to preserve the total signal strength from inputs to outputs, taking some tiny portion for its efforts.

Memory.

Memory of an agent as a whole is constructed from the charges in the past stored within each neuron. A past charge does not have a time reference, but the mind can scan all neurons charges at once at some particular time stamp. It is still an open question of how these past charges participate in the decision process on an agent.

A long way to OS X

One beautiful morning I decided to try out Mac OS X on my laptop (MSI EX625). There were several reasons:

1. Apple hardware is too expensive.
2. The OS quality is outstanding, while the price is just 40$.
3. Need to try developing for Apple's mobile devices.

First facts I discovered was: "it is possible" and "there is TonyMac boot loader". Alright, I ordered a disk from eBay for 30$, starting to prepare myself mentally for a change.When the disk arrived, I was eager to try out tony's boot loader. Chameleon appeared on the screen, inviting me to start the installation. The first try failed - it couldn't reach the installer window, hanging half the way through.

I searched forums and started trying different boot options. Nothing actually helped. There were several errors on the screen, what confused me a lot, especially because an actual error causing the hang was not mentioned... I tried all possible ATI versions of iBoot (4+) and even PcEfi with no result. I've posted on the forums, waited, searched, but hadn't gotten any replies...

After heavy surfing I found out the actual reason - ATI Radeon Mobility video-cards are not supported! There was a chance that drivers could be released soon, but I couldn't wait, especially an undefined amount of time. I decided to exchange laptops with my wife, leaving old hard-drives. She didn't stop me, so I started to disassemble both laptops immediately, with a help of a small screwdriver. My new laptop was ASUS M51SN, with ultra-compatible GeForce 9500M on board.

Both Linuxes survived this change with honor: my Arch required to change the video driver, while Dina's Ubuntu worked perfectly like nothing changed (though, she updated the driver later). Mac OS X still didn't run out of the box, but required the following options: '-x' for safe mode, 'cpus=1' for proper cpu cores detection. That was sufficient, but I patched the BIOS using a custom version to enable proper CPU detection (the BIOS built-in update utility is awesome!).

Hello, the freaking installer! It wasn't easy at all to get you on the screen! I was happy for a short moment - the installer didn't like my MBR disk at all... Apple supports only GUID & Apple partition tables, intentionally ignoring MBR. There was a solution - to use the external USB disk/flash: to install on it first, and copy the partition to the hard drive directly. My 4Gb flash wasn't enough, and the official website states 5Gb requirements. Borrowing a 8gb USB flash was not enough as well, because the minimum installed configuration took 8.2Gb, what a surprise!

That was a time to sacrifice my precious 1Tb external USB drive. There was no space to backup to, so I decided to trick the installer a bit. I moved the actual data partition +20Gb from the start of the disk, leaving the empty space for the Mac OS image. I backed up the MBR together with first 100Mb of the FAT32, just to make sure the file system structure is saved. In the installer, I re-partitioned the drive into GUID 2 partitions : first 20G for Mac OS and all the rest. The installation went successful, yaw!

I copied the partition to the hard drive and was even able to boot from it: in a safe mode, using the same cd loader. Moreover, putting the old MBR back on the external HDD recovered it instantly, leaving me with a feeling of the epic win :) However, the OS X file system became corrupted each time I booted, forcing me to rewrite it from the image and look forums for an answer...  Posting again and asking for help in the OSX-86 IRC didn't help, leaving me in frustration again...

The sorting of my MBR seemed to be a good idea, if not for the corruption problem, but at least to make the boot loader work on the HDD. Fdisk did the job, but MBR was somehow damaged after that. I looked for a rescue and accidentally issued 'dd' command on the saved 100Mb file from my external drive backup... That was a bi-i-ig mistake, and it seemed to be the worst situation ever: the partition table together with my ext4 file systems were destroyed, so I had no OS to boot and no data to operate!

That's where smart hackers come into play. Asking again on the IRC, I attracted the attention of 'aschar', who listened to my problems and described the way out, in detail. The solution was to use a much less known Nawcom boot loader, that recently gained MBR support. In a day I managed to rebuild the partition table, entering some numbers by hand and using a couple of hack tools (look at testdisk!).

Using the PartedMagic liveCD I prepared the case-sensitive HFS+ partition and started Nawcom's boot loader. From now on, everything was smooth. The boot loader didn't require the safe mode, allowing me to install OS X right into the MBR partition I provided. After reboot, the OS loaded correctly, allowed me to update it to 10.6.4 and install MyHack tools from the same CD. No file system corruption, good driver support (though, sound & wifi are in progress) - and now I'm a happy owner of the Mac OS X laptop!

The end of story. The moral is:

• Always look for alternatives! What is more popular is not always of better quality.
• Always backup, at least MBR! Be careful with 'dd' and 'sudo' in general. Think, then do.
• Don't give up!