Neural networks emerged inspired by the human brain, so the operational mechanisms of human society also have various similarities and connections with neural network training.
This year, I dabbled in many other disciplines. Most were superficial explorations, but I still gained tremendously. Each time I dig into a new field, I habitually find connections between the new discipline and old ones. Particularly, I prefer to view this knowledge from the perspective of my major. For example, the connections between human society’s operational mechanisms and training neural networks.
Every person can be imagined as a computer:
They have a brain serving as CPU and memory; a cerebellum as peripheral processor; a beating heart as the power source for the entire machine; they have various input devices: eyes as cameras, ears as microphones, nose as gas-phase component analyzers, tongue as solid-liquid phase component analyzers, skin as pressure-temperature sensors. They have various output devices: muscles as actuators, vocal cords as audio signal output (though these should also be muscle-controlled). Muscle-controlled posture and expressions serve as human displays, while vocal cords serve as speakers.
With input-output, computation-storage-control, and other components all ready, as a person, their hardware is prepared. However, having only flesh as hardware is not enough to constitute a complete person. Software - that is the soul of humanity. All our behavioral patterns can essentially be abstracted as “receiving stimuli - making responses.” Software is such a specific processing mechanism: it matches specific inputs with outputs, completing the transformation and mapping from sensor signals to actuator actions.
From crying babies, humans begin continuous learning - the process of establishing these mapping relationships. First, we learn a language, establishing the most basic finite state automaton model, establishing basic rules: for thinking. Further, based on this language, we accept social cultural influence - this is installing the operating system. So-called culture - might as well understand it as a set of specific behavioral patterns that provide universal norms and standard interfaces for all behavioral patterns of people living in the same society. Accepting cultural influence means following such communication norms, using the same standards to operate. Of course, culture only provides suggested standards (RFC). These standards we call morality and values. What provides mandatory standards required for operation is law.
With hardware and basic operating system, the computer can now run. But just as neural networks need training before use, so do humans. A child’s behavior often appears very unstable: two similar inputs may produce vastly different results. But through training and learning, this instability gradually disappears: after ten to twenty years of training, people become mature and steady. If emotional states are included as input factors, then the situation where identical inputs produce vastly different results should rarely be seen in normal adults. This is entering a stable working state.
However, after installing the operating system, real tasks are always completed by application programs. A computer’s value is reflected in the work it can perform, and the work it can perform is determined by its software. The process of receiving education is installing these software programs. It’s establishing a neural network for implementing new functions and training it.
For some basic applications: like Paint, Sound Recorder, Notepad, WordPad, Media Player. These most basic software we learn at home and in elementary school. More advanced Word (language), Excel (mathematics), PowerPoint (common knowledge) are completed in the pre-university education system. For more advanced professional software: antivirus software (medical), Matlab (engineering), Latex (science), Acrobat (liberal arts), power management (agriculture), network applications (business) - these software we acquire in university. But just as universities vary in quality, software companies’ products are also mixed. This belongs to poorly trained, poorly designed neural network models.
Some computers, after installing this software and graduating from university, can formally enter computational work. But another part of people further master more advanced software: knowledge for creating software. Writing new software, creating new knowledge. This is pursuing a PhD, taking the academic path. Creating a neural network that designs neural networks, establishing a self-referential system.
For each computer, their strategies for choosing which software to install are different. Some computers install many various software programs, each using only simple functions, but combining these software programs together creates infinite possibilities. This is breadth-preferred learning, characteristic of scholars. Some computers install only specialized software, but highly customized, using powerful functions with ease - depth-preferred learning, characteristic of experts. This happens to be the difference between MBTI’s fourth dimension J and P. Personally, I think the expert type is superior, but in actual execution, I tend to use both strategies in combination: scholar type for undergraduate, expert type after graduate school. Because the learning curve for almost every discipline follows logarithmic or logarithmic-s curves, according to diminishing marginal returns, efficiency in early learning stages is significant. All software has commonalities, and simultaneous learning can reduce learning costs. At the same time, it’s more likely to create incredibly powerful combinations: like Resharper with VS, Everything with TC, etc. Of course, a more effective solution is: reduce daily downtime for maintenance to simultaneously address both depth and breadth.
Whether in research or work, one thing is certain: skill proficiency gradually increases with time, and experience accumulates through labor. This is a parameter optimization process. Some people learn fast but not solidly - high learning rate α; some learn slowly but solidly and stably - low α. Some learn both fast and well - good convergence algorithms, strong learning ability, high IQ.
Human learning speed also decreases over time - an aging process, a stabilization process. Old professors have low α values; their knowledge has converged to specific points, efficiently adapting to stable, subtle environmental changes and quickly providing high-confidence outputs to inputs. Young people have high α values, unstable outputs, but are very likely to break into new fields.
Humans have a special characteristic compared to machines: emotions. What are emotions? Emotions should be psychological reactions to external stimuli. They can actually be viewed as fluctuation factors, disturbance factors. However, emotions cannot exist as simple bugs. In my view, emotions are extremely complex and advanced reaction patterns and rules, survival strategies selected through billions of years of natural selection, mature neural networks trained through countless brutal competitions. When we face emergency dangers with no time for analysis and decision-making, they provide high-speed, efficient exception handling mechanisms - fast approximation algorithms. It’s just that these methods are often abused.