# Generalization ### General Definition A generalization is a defined as: > **Generalization**: A broadening of application to encompass a larger domain of objects of the same or different type. Why do we perform generalization? *The reduce complexity by replacing multiple entities which perform similar functions with a single construct*. We can get slightly more technical and take a [quote from wikipedia](https://en.wikipedia.org/wiki/Generalization): > Generalizations posit the existence of a domain or set of elements, as well as one or more common characteristics shared by those elements (thus creating a conceptual model). As such, they are the essential basis of all valid deductive inferences (particularly in logic, mathematics and science), where the process of verification is necessary to determine whether a generalization holds true for any given situation. An example will help illustrate this. Consider a cake, cookies, and ice cream. Each these items is it's own abstract entity. However, we can establish a *common characteristic* between these items. In this case the common characteristic may be that the majority of their content is made up of sugar and butter. Items that intrinsically have this characteristic can be said to belong to a set that contains all other items with this characteristic. This set is that of all **desserts**.  We can see that generalization is indeed a form of [Abstraction (Computer Science)](Abstraction%20(Computer%20Science).md). We have *abstracted* a common properties of specific desserts into an abstract "dessert". We can now ask our partner to pick up a "dessert" on their way home, without needing to list out all possible items that could fall into that set. ### Computer Science Generalization is important in computer science because it allows us to find common themes and patterns, which reduce the overall amount of code. We can build off of the [book example](Abstraction%20(Computer%20Science).md#In%20computer%20Science). Of course there are other things in a library that one can borrow. We can call the set of all those objects `Borrowable`:  We can write generalized logic that applies to *all* `Borrowable` objects. For instance, they may have a `lend()` method, as well as a `last_checked_out` property. So, instead of needing to write the same `lend()` method for all classes, we only need to write it *once*! ```python class Borrowable: def __init__(self): self.last_checked_out = '' def lend(self): # Set last checkout out to today # Set user idea who has it # Set when it needs to be returned # Etc class Book(Borrowable): # we have access to Borrowable methods now class Magazine(Borrowable): # we have access to Borrowable methods now class DVD(Borrowable): # we have access to Borrowable methods now ``` ### In Learning Theory > In the compressive phase, exploration will dominate and randomization methods facilitate these explorations. In this regime, the gradients [carry negligible information](https://arxiv.org/pdf/1703.07950v2.pdf) and thus the convergence is extremely slow. This is where representation compression occurs. The elusive goal of Generalization is achieved through the compression of representation. We can explore many interpretations as to [what Generalization actually means](https://medium.com/intuitionmachine/rethinking-generalization-in-deep-learning-ec66ed684ace), but ultimately, **it boils down to the shortest expression that can accurately capture the behavior of an observed environment**. See more [here](https://medium.com/intuitionmachine/exploration-exploitation-and-imperfect-representation-in-deep-learning-9472b67fdecd). --- References: * [Abstraction and Generalization](https://web.archive.org/web/20180328151725/http://www.emu.edu.tr:80/aelci/courses/d-318/d-318-files/plbook/absgen.htm)