psy (27Aug2006): Zola posted a rather large list of questions regarding Yake's design and code. I pasted it into the Wiki as it's easier for me to add answers. The forum thread is still useful for detailed discussion.
I will add answers to this page as I go.
psy (27Aug2006): Zola, it's a pleasure to know that you spend your energy not just going through Yake's code and design but that you compiled and published your thoughts and questions. This is very important to me and other Yake developers! And the first signs of changes due to your feedback are already in SVN HEAD (0.6.0-dev). :)
It's here: Questions and Thoughts
Hello,
I've been reading Yake's code, and I have a few questions. I may call into question some of Yake's design, but doing so helps me to understand how to use Yake and why things are the way they've come to be today. I refer to parts by their files as there's some naming confusion.
To start:
1) Since Yake has a relatively complex hierarchy of subdirectories and components, why the separation of /yake/yake and /yake/src? While working my way through the code and examining parts, this made it cumbersome to delve. I had at all times to keep two locations open and synchronously browsing corresponding parts of /yake/yake and /yake/src. This kind of split seems good for a simpler layout. But for yake, there's a big divide between the two sides– corresponding trees must be maintained.
2) What conventions exactly, in plain terms, should the pch, prerequisites, and yakeComponentname.h adhere to? Which includes, defines, or otherwise should specifically be in which files? I've seen different parts of Yake doing very different things with these files.
3) Overall, file naming needs cleanup. Taking example from point 2), Some are named pch.h, some yakePCH.h, some prerequisites.h, some yakePrerequisites.h, and some hand over to common.h.
4) Why reimplement yake/base/math/yakeMersenneTwister, instead of relying on boost::random? Boost is already a dependency for yake, and provides mersenne twister based random numbers.
5) Why is yake/base/math/yakeAsmMath needed, and when would this be beneficial? When can't we trust our compiler's implementation?
6) What was yake/base/yakeCommandInterface intended for? It's used nowhere.
7) yake/base/yakeString: why not boost::string_algo?
8) yake/base/templates/yakeNiftyContainer is used nowhere in the sourcetree; yake/base/templates/yakeFunctor is commented out and replaced by boost::function. When will the gunk be cleaned?
9) What is the purpose of the STL wrappers in yake/base/templates? Ie, yakeDeque and yakeVector. There's so many things like this which make Yake larger and less simple, but don't seem to add much. It seems to be suffering from kitchen-sink syndrome. Something like yakeFastMap is understandable, but something like yakePointer's abstraction on top of boost::shared_ptr is not. Why make Yake even more complex with all this?
10) yake/base/yakeSignal's signal abstraction does not appear to be used in a number of places throughout Yake. yake/ent/object for instance, explicitly does typedef boost::signal<…
11) Why has the registry been created, as opposed to using a more traditional OO approach of ie, ISomethingInterface* = new ISpecificSomethingInterface?
12) Why not use the reflection system's createInstance instead of the registry? It can instantiate types by name.
13) What does yake/base/templates/yakeManager do? Is this a refinement of some pattern?
14) yakeManager seems to be some of the worst code in Yake. I think I read this code three times and couldn't figure out what the author was even intending to do. So what's it do? The policies are jumbled up. Why would a vector policy even be there if there's no getObject or getIdentifiers for it?
15) yake/base's components are spread out over various namespaces. Some are in namespace yake (ie, yake/base/yakeTaggedListenerManager), while others are in yake::base. Also, what is the distinction between yake::base::templates and yake::templates, as used in ie, yake::templates::create<>? This all needs to get standardized.
16) How are the ListenerManagers supposed to interact with the rest of the system, and how to best make use of them?
17) Regarding yake/base/yakeCriticalSection, yake/base/yakeLock, and the accompanying files in yake/base/native, boost::thread already abstracts this by implementing Mutexes with Critical Sections when on Windows. I can't really tell if Yake is using boost::thread or not, but if it so, all the CriticalSection stuff is unneeded.
18) How does one decide whether to use yake/base/yakeLog or yake/base/templates/yakeSmartAssert's debug messages? They both do much of the same thing. The distinction between these two is a bit blurry.
19) yake/base/prerequisites needs STLport support fixed up. Various of the platform includes have stlport checks commented out, and HashMap explicitly #defined to use libstdc++'s __gnu_cxx::hash_map.
20) There seems to be a bit of overlap and duplication in the yake/base/mpl section. For example, why define yakeSelect instead of using mpl::if_c? And half the files are duplicated, ie both yakeInheritLinear.h and inheritlinear.h exist.
21) Why is yake/base/mpl/yakeCast in the MPL section? This goes hand-in-hand with safe_delete, which is in yake/base/templates.
22) What is this very strange split/join stuff in yake/base/mpl/yakeCast? What is it doing?
23) What is yake/base/mpl/yakeCast's getBasePointer for? It has a funky implementation.
24) yake/base/mpl/yakeAbstractFactory: An abstract factory may require MPL, but it is not MPL. MPL is for the most part compile-time metafunctions and containers; an abstract factory is higher-level, and seems it should simply be a part of base.
25) The decision to keep Ogre loosely coupled seems dubious. I can clearly understand writing a layer which “tells Yake how to use make use of Ogre,” but see trouble in asking the user to “Do certain Ogre things through Yake's interface, but other Ogre things directly with Ogre.” Invariably we'll be juggling between using the Yake interface and using Ogre's interface. We immediately run into the problem of converting Yake Vector3s to Ogre Vector3s. Ogre is huge and trying to come up with an intersection for commonality between Ogre and potential future renderers seems impossible or at best heroic but hopeless. It makes sense of course to use Yake's API for composite abstraction of Ogre parts, ie the Model or loader systems which can in turn use Yake's Ogre layer. But there's a lot of potential here for future features in Yake which would rely on Ogre. Worrying over Ogre's coupling might keep a lot of these features from ever materializing. At the surface, Yake's application framework could provide nice ways of accessing Ogre's base objects.
26) At the library-user level, what kinds of things might we use reflection for besides as a unified means to serialization, script binding, network replication, and GUI construction?
27) Why are hardly any parts of Yake itself using the reflection system's basic provisions such as the CLASS macro?
28) What's the difference in roles between reflection's 'Class' and 'Object'?
29) What is a reflection “event”?
30) In yake/reflection/class, why did you choose to use std::list's for storing members?
31) Why does each Class object have both a PropertyList and a _PropertyMap? Why not just iterate over the map?
32) How about templated conversion operators in things like Field, so that we don't need to use get<>?
33) Having to explicitly provide the type at compile time when get<>ing a value out of a Field is a problem. It should have a memberfunction which can construct and return a boost::any.
34) Why were the register structs needed, such as 'struct register_field' instead of static functions?
35) Why do the getType's, ie in yake/reflection/field's 'const char *getType() const' return strings instead of type_info? I'm guessing that it's because type_info elides reference types. If this is the concern, a getTypeInfo() should be provided as well.
36) yake/base/yakePrerequisites has a #define YAKE_DECLARE_CLASS(name) static const char* yake_private_currentClass() { return #name; }. This is used throughout Yake, but yake_private_currentClass() is never actually accessed by anything. It overlaps with reflection's Class::getName(). Why not unify here on the reflection system?
37) Why does yake/reflection/bind_serialization use boost::serialization at all? A big part of boost's serialization library is the setting up of a way to define the members which need to be serialized, without the presence of reflection in C++. It seems that custom serialization should be trivial when reflection is present, and shouldn't require boost's serialization library. Serialization is a different beast when reflection is present, and boost::serialization mostly solves a problem which Yake circumvents entirely.
38) The reflection examples are using obsolete versions of reflection; RXO_DECLARE, etc. Even the main.cpp in yake/src/reflection really doesn't show you how to simply use the reflection. It starts off with properties but gives you no base understanding of how to query the members in a class. What is needed is a single file which builds upon each prior example, showing each base feature of reflection.
39) What is the relationship between yake/prop and yake/reflection/property?
40) I see that reflection properties are compile time, while yake/prop properties are purely runtime. Perhaps 'prop' should be renamed.
41) yake/base/templates/yakeProperties: If this has been completely deprecated, nuke it out.
42) yake/prop is using RttiClass so that it can keep a class registry for property inheritance. Why not use the reflection system's Class hierarchy? yake/prop/class_rtti's DECL_CLASS_RTTI seems to directly overlap with yake's reflection.
43) Why is only yake/prop using RttiClass, and only yake/ent using yake/prop?
44) yake/prop/prop_holder's PropHolder is template-parametized by the class which contains it; 'template<typename T> struct PropHolder' .. but T is unneeded– Why is it this way?
45) PropHolder overlaps with yake/base/yakeParamHolder. Why not a central implementation of this concept that's both lightweight and generic enough to work for both?
46) Properties in the sense of yake/prop seem to be the idea of runtime variable groups. The property_tree library was recently accepted into boost. It provides a generic tree of properties with modular serialization support. The property_tree can be read/written from XML or any number of formats, and you can subclass to provide your own. This does everything that tinyXML does, but without being tied down to a specific fileformat. yake/prop/prop_holder does not seem general enough. I see the need for a single standardized container which can store trees of property objects or other kinds of data, and is serializable for network transmission or file loading & saving. Such a property tree system can do the job of yake/prop, replace yake::ParamHolder, represent the hierarchies for yake/loader, and is also useful to library-users for storing game data.
47) What is the relationship between yake/msg, yapp/msg, and yapp/base/event? What's what?
48) I'm guessing that Yake presently has three different messaging systems in its sourcetree, two of which are obsolete and deprecated. For this reason I'll make no attempt at understanding yapp/msg or yapp/base/event. I can only guess that yake/msg is the one which is not obsoleted at this time, considering it supports both the concepts of target and source, while the message system in yapp/msg does not. Is this right? Interestingly enough, yapp/msg's implementation is huge and yake/msg's is tiny.
49) In what ways should the messaging system be used in the context of a game or higher-level project?
50) How does yake/msg coexist with signals? In what situations are we to pick a signal instead of a message? Why not replace signal altogether with the messaging system, such that both queued and unqueued messages are possible?
51) yake/msg uses an odd #define for YAKE_MESSAGE_NAMESPACE. Is this convention now intended to be used throughout Yake?
52) In yake/msg/message, why have you chosen to use void*'s as message source/target identifiers? I'm guessing it's so that the identifiers can function as both integral object numberings, or to directly hold the addresses of objects so that no numberings are needed.
53) In yake/msg/listener, why are you casting 'fn_( static_cast<const MsgType&>(boost::any_cast<MsgType>( msgbase.value())) );' to a reference?
54) What about messaging between threads, and the necessary locking? How does yake/msg fit in here?
55) What sources, research, books, links influenced Yake's messaging system? I'd like to read more about messaging systems, but links are difficult to find on google because searching for “messaging”, “message posting”, or “event systems” returns a bunch of bogus results.
56) boost::asio (the recently accepted Asynchronous IO library) provides an implementation of the 'proactor' pattern via an io_service which can have functors .post()'ed into it. io_service has three methods of posting: .post(), which pushes the functor into the queue, .dispatch(), which executes the functor immediately and then returns, and .wrap(), which returns a function object wrapping the passed-in function object that will dispatch to the io_service when invoked. boost::bind is used for passing specialized messages. The cool thing about boost::asio is that a pool of threads can simply poll the io_service, and when a thread is free it will handle the next available functor in the queue. Could this apply to Yake, for a messaging system? yake/msg/router is synonymous with boost::asio's io_service. yake/msg appears like a combination of the io_service and signals. Yake's router is parametized by type-being-sent; the type itself determines what to send to. But io_service doesn't support such things. As they're shooting to get ASIO into C++'s tr2, it should be getting a bit of code love in the future.
57) What is the difference between yake/object and yake/ent? Where do we use an Object, and where do we use an Entity? I've read the brief description in the recent announcement on the news section, but still do not understand their roles.
58) The organization of yake/ent and yake/object is a confusion. 'Entity' isn't in the expected yake/ent/entity.h– it's in yake/ent/object.h.. and 'Object' isn't in yake/object– it's in yake/ent. It seems that yake/object should be renamed to yake/objectid, yake/objectid's files renamed to denote that they're ObjectId related, and the definitions in yake/ent be separated into appropriate .h's. Ie, ObjectContainer → ObjectIdContainer.
59) Yake has two different concepts of Object. One in yake/ent/object and the other in yake/reflection/yakeObject. They're totally different classes but with the same name. As Yake uses namespaces throughout, it's simple enough to think “one's a reflection object” and “one's an object object.” But the idea of an object is a fundamental enough building block that we need to be able to think of it in terms of only one definition. There's no problem with common class and function names existing in multiple namespaces, but Object is an exception. This also makes human communication easier.
60) What is the differentiation of roles between yake/ent Objects and yake/reflection Objects?
61) yake/ent/component, component_holder: What are components?
62) yake/ent/prerequisites.h has a strange ClsEntry struct and a 'typedef object::ClassAndObjectIdManager<ClsEntry> ClassAndObjectIdManager' but it is not used. What's this for?
63) What are some examples of things we'd create ObjectListeners for?
64) The concept of yake/object/ObjectId's ClassId conflicts with the idea of a ClassId as defined in yake/base/yakeUniqueId.
65) What is the relationship between yake/ent/object_mgr and yake/object/ClassAndObjectIdManager? Or the difference between an ObjectManager and an ClassAndObjectIdManager?
66) yake/ent/object_mgr and yake/object/ObjectManager conflict. Which one to use?
67) Why are the implementation files for yake/object/ObjectId, yake/object/ObjectManager, and yake/object/ClassAndObjectIdManager all commented out? What's been deprecated?
68) How about the idea of a connection between the messaging system and the Entity/Object system, so that we can send messages to ObjectId's and ObjectId groups?
69) What's with yake/plugins/scriptingLuaBindings– Is it just an example?
70) Is yake/reflection/bind_lua's complex implementation there to do anything besides handle luabind's inability to understand the type of data that Yake's reflection is able to provide?
71) Scripting events: What is an lua/scripting event, and how does this connect with reflection events? It seems to provide a way for callbacks to a script, but I don't see why it needs the reflection system.
72) Is yake/model done being refactored/rewritten? If not, how will it need to change?
73) Can Models hold Models?
74) yake/model seems to have no connection with Yake's Object/Entity system. How do these come together?
75) Is the idea here that we create a Model which IS-A Entity? Ie, a Model that contains an IEntity, and inherits from Entity.
76) yake/yapp/vehicle makes no use of the Entity/Object system. How does this work? It only uses Model and Movable. I've noticed that throughout Yake's code, 'Movable' is acting as a baseclass where other engines would have something named “Entity.”
77) Why is an IAvatar a ListenerManager<>? Can you explain how this works?
78) The physics abstractions are built for signal slots, but not message listeners. Isn't this a good place to support messages, as we might decide our physics' router should processMessages() at a fixed rate?
79) I've only had a cursory look at Yake's Net component so far. Please read this thread I've posted on the Enet forums at http://lists.cubik.org/pipermail/enet-discuss/2006-July/000618.html and tell me your thoughts.
80) Has Yake any plans for 'interest list' support in the Network component: replication only of entities which meet some criteria for range or visibility?
81) What is the present status of Yake's GUI abstractions? I've read some posts to the effect that there's multiple Yake GUI components, or that the Yake GUI isn't being maintained at this time.
82) It's important to choose descriptive names for plugins so that we can understand what they do. In yake/yapp/plugins, there's ceguiOgreRendererAdapter/ renderer_adapter_cegui_ogre/ and gui_cegui/. I think that more descriptive names are needed which clearly define their roles in english.
83) What is the meaning of the letters “Rt” as in yapp/raf/yakeRtApplicationState?
84) Yake's samples are not only “not so good”, but they're an exercise in confusion. They're downright misleading. Their existence is worse than not having anything at all. Take a look, for instance, at src/yapp/samples/base/objects/yakeDemo.cpp. It implements some reflection attempt that has no bearing on anything in Yake. The majority of other examples are using deprecated code, nonexistent code, do not compile, or will send the user on a wild goose chase. These need to go.
85) Sourcetree layout is confusing for all the Samples. Ie, src/yapp/samples/base. Why is vehicle there..
=Bold Text
86) What system in Yake is used for registering subsystems to process each frame?
87) Does Yake have plans to handle the spawning of threads for scripts which go into a loop? Say that a script processes infinitely, spitting out position updates. I cannot rely on on_tick; a single script can stop the whole server process.
88) What about Tasks, a-la yapp/base/yakeTask, but for tasks which should spawn threads? Does Yake have some way of handling this and the necessary task dependencies / thread synchronization required for posting back results? If not, what can be done here?
89) Someone writing an app needs to store, well, central settings for that app. There is a need for a “configuration system” or “settings registry” which is loosely coupled, has hierarchical storage, can emit signals when a setting has changed, and can serialize. The loose coupling is needed so that our whole sourcetree does not need to recompile when we add a new config option, as many parts of the code connect to such a settings registry. This could use the same system as outlined in point 46).
90) Does Yake have plans for a higher-level system that is aware of Entity locations and can load/unload objects automatically as the player moves around a world?
91) The amalgam of base's ClassId, ent's ClassId, ent's ObjectId, ent's Object, reflection's Object, reflection's Class, and prop's RttiClass particularly troubles me. Can you explain, in one place, the rationale for how they all come together?
92) Has Yake any plans or present way to do background resource loading?
93) Yake has some excellent ideas, and in isolation the individual components are good. The present incarnation of yake/msg comes to mind for its clean and clear implementation; it does a lot in a few lines of code. But as a whole Yake exhibits no coherent design vision. Yake, at this time, appears more like a sandbox for testing out engine ideas than an engine; a loose collection of contributions rather than a tight composition which works in a specific way. An engine is something which is tightly engineered in order to drive a vehicle forward optimally. The codebase is an elephant-graveyard of deprecations, overlaps, and vagaries. There's no reason to let TRUNK rot with decaying code; CVS/Subversion's time-machine-like abilities allow us to freely look back on prior versions of components. Yake needs consistency. The gestalt of all of these factors is that a developer who would like to use Yake is facing a steep starting curve to even figure out what parts he's supposed to really use. It appears that the codebase is being maintained by multiple people with no central arbiter to assimilate everything into a coherent form. People commit new parts, but those parts aren't tooled and standardized for the overall design. That's not to say that things can't get better. Yake is in a great position to be cleaned up, due to its modularity throughout. I grant that sometimes it's better to try out a bunch of different ways, to “just do something” instead of doing nothing, to not fret forever over the-exact-perfect-way-to-do-it. This can get some motion going, and the ensuing chaos can later be wrangled– it's a valid development technique. But a decision should be made upfront for how the side effects of this development strategy will manifest. One can let it produce a rotty-sourcetree, or instead it can be shifted/transformed into we're-quite-often-breaking-the-interface-and-systems-that-were-there-previously-just-up-and-disappeared. The latter is superior because it allows a single release at each stage of the game to be coherent.