"If your goal is pretty focused it is likely it can help you a lot. If your goal is super complex with many interleaved parts then it will lose context as some point and start hallucinating."
Perhaps the most complex thing I'd like to achieve is managing an aquaponics setup in a greenhouse. I need to determine what metrics need to be routinely attended, PH, nitrates, flocculants, flow, water level, temps, status of filters, and checking these things at several points in the system, several chemical burdens in the incoming water supply, and then mitigating issues when triggers get pulled. These are intended to start small, with a couple sensors, for the first three or five, and manual mitigation measures (and manual confirmation of the metrics reported), but to gradually expand as sensors are added, and software is updated to track new metrics.
This process is expected to be pretty modular and gradually expand, so I think overly complex development of code in a single chunk is unlikely to trigger hallucinations, and that's what the manual confirmation of the metrics is intended to detect. Eventually I'd like to add arm(s) on a rail that can add deflocculants, change filters, even pick and plant crops, and hopefully fish and crawdads. Because I don't expect to start with the turnkey management system, but intend to create one that I can share publicly so that people can simply install the necessary hardware, biologicals, and software to run it all, and enjoy the produce and menu additions a greenhouse aquaponics system in given conditions will generate.
"You need to know enough about what you are doing to be able to read the code and say "this is not right".'
I don't, but I can gain that competence as needed. I taught myself to open Notepad and code up a (simple) website in HTML at one go back in the early 2000s, so I expect to be able to gain basic competence in at least Python (enough to ascertain some other language is probably more suitable, LOL). But I sure don't have that competence at present. I don't even know why it's hilarious to code Linux device drivers in Java. I only know programmers think so.
I don't think the main hurdle in this development process will be AI, but my own ability to consistently manage the system in developing conditions, as I expect significant political and economic changes in the next decade.
"They always speak with authority as though they really know what they are talking about even as they are feeding you bullshit."
Since that is one of the primary reasons I limit my interactions with actual people, it's why I think I will most struggle to work with AI. That and the obsequity will quickly dispel my capacity to overcome my aversion to such pathological manipulation. IME I quickly wear out my welcome with people that act that way by cutting to the chase they are desperate to avoid, which is why my political influence is actually a negative quantity locally. Maybe that approach will succeed with AI that can't refuse to take my calls?
"...a TileSystem that enables combination of prefabs as tiles, traditional terrains, shapes with varied texturing and shaders, support for cube, marching cube, and dual contouring voxel systems..."
This is an entirely different use than I intend, yet I see the similarity in the way managing these separate mechanisms "...unified into a single system that makes them all play together nicely." is exactly what I seek to do with ensuring the aquaponics system is functioning nominally. Perhaps since creating virtual environments and running a greenhouse both involve managing metrics in 3D space.
"...give it a shot."
I don't see anyone else doing it, as you don't for your usecase. In fact, I can see some overlap between what you're doing with what I want to do, because I want to enable people to come up with a site specific design that will enable constructing a turn key aquaponics system, and the exact processes you're coding will be necessary to design a working system, at least exchanging NPC's with maintenance procedures and with certain life support requirements that may only apply to your environment designs for exotic environments, like for extraterrestrial or extreme environments.
Given my limited coding experience - essentially none outside basic HTML decades ago - Paduan will not be snatching pebbles from the master's hand any time in the foreseeable future. However if you can see how some code you have developed might be adaptable to tracking sensor data for the systems I want to monitor, and eventually maintain, any specific advice and consent would perhaps be miraculous in facilitating my success. When I have a functioning system I seek to manage, would you mind if I inquire of you regarding adapting your code modules in that way?
RE: Conversations with AI that I could have (perhaps should have) had with people...