How difficult can development get?

[cguy]

However, I am battling in my mind to logically arrive at difficulty rating for my profession.

There is no single difficulty rating for software development - the scope is way to wide. Brick layers, carpenters, engineers and architects all build skyscrapers.

a) How difficult can development be in terms of problem solving? Bring out the really big guns here such as high scalability infrastructures, building frameworks and middle-ware for the masses to taking the lead on technologies and where they go.

There are extremely challenging problems out there. Off the top of my head:
- computer vision to query for people or objects, from images or videos, possibly with real-time constraints
- generating 3D models from sets of images or videos
- algorithms to predict and/or trade on the stock markets (complex algorithms/maths and a mistake can cost hundreds of millions of $)
- simulation of physical processes for designing anything from radios to engines
- massive data processing and/or visualization
- automated data mining, feature extraction, machine learning, prediction, etc.
- robust OS kernel and driver development (imagine if a single NULL dereference can cause 100 million machines to crash 10 times each)
- system software for exotic architectures
- improving on state of the art ray tracing techniques for more realistic movies, or real time rendering
- porting algorithms to new hardware
- working with hardware designers to build hybrid software-hardware solutions to computational problems
- developing scalable distributed versions of existing algorithms
- developing new algorithms (or modifying old ones) for new hardware architectures (Xeon Phi, GPUs, custom (FPGAs), Cell, etc.)
- improved compilers (dynamic profiling, on the fly recompilation, auto-vectorization, etc.)
- performance tuning in general (massive topic, with a crazy amount of complexity involved when taken to extremes)
- algorithms in general: optimization (in the "function minimization" sense) under general constraints, computational geometry, graphics, graphs, sorting, number theory, numerical integration, search, etc.)
- etc, etc, etc.

b) What differentiates a mediocre developer from a great developer? Rather, what would make a developer unique or invaluable?

Obviously those who are the top of the fields I listed in (a) would be great. If you are a world leader in any of these, or are highly competent in a rare subset of these, it would make you relatively unique.

As for "invaluable" what's really important to understand, is that this is a demand and supply issue. Ideally (for you), the demand would be "from everyone", and the supply would be "just you" - in general, the harder a field is to understand, and the more education that is required to master it, the more likely there is to be a smaller supply. This doesn't guarantee demand though. It's also important to realize that the way a lot of people attain value isn't by having some elite absolute level of skill relative to the rest of the world (sure, this would help a lot, but is far less common), but rather by developing unique domain knowledge within their business.

c) Do you think, in terms of pure problem solving skill and talent, that a developer could be to technology what a doctor can be to a patient? Do you think the problem solving levels are on par?

I am fairly certain that there are many software developers and engineers that far outshine doctors in problem solving skills. Not ever doctor is House (from the show, not myBB) - many follow a very simple diagnostic flow chart, and have a relatively superficial grasp of the underlying concepts (same can be said for many developers). I am both a developer and a doctor (no, not the kind that helps people... ).

So in essence, is there a limit to the problems IT has at a given time? In my mind, it seems everything can be solved with software and

logical design. Granted there is the limit of hardware (processing and memory), but that's more a resource limit (when solving developmental challenges). If you throw enough money at it (resources), it will be possible.

That's a bit like saying that any problem involving numbers can be solved with mathematics. The only limit is the set of problems that the world currently has. Software is just a tool to manifest the solution - the actual solution is an engineering process and conceptual methodology.

So what is impossible? What are the 'mysteries' of technology (development wise)?

Trying to get 'make' to behave correctly on large projects.

 

[gkm]

I do not think it is really relevant to compare software development to medicine. In software development, skill levels can easily differ by two orders of magnitude between the best and worst developers. I think this is closer to creative disciplines. For example, I can barely draw a stick man, but there are people that can do a beautiful painting in a couple of days. Same with development, where some people are probably 100 times better than some others.

Development does have some strong engineering skills and techniques required to be effective, but that alone does not make a good developer. For me, this is that same as for example, somebody that makes statues, who can make good molds, is significantly better than somebody that does not know how to do that. Like many creative disciplines, good development is a combination of good creative skills and good "engineering" skills. And the same as for other creative disciplines, to be really good, you have to be "driven" and have to have talent. Just like with many creative disciplines, people can be trained to become reasonably good at it, but for the final step, talent and drive is needed.

The range of problems range from writing hello world (http://en.wikipedia.org/wiki/Hello_world) to making a car drive by itself (http://en.wikipedia.org/wiki/Self-driving_car) etc. and beyond.

... So in essence, is there a limit to the problems IT has at a given time? In my mind, it seems everything can be solved with software and logical design. Granted there is the limit of hardware (processing and memory), but that's more a resource limit (when solving developmental challenges). If you throw enough money at it (resources), it will be possible.

So what is impossible? What are the 'mysteries' of technology (development wise)? ...

It has been proven that some things cannot be solved, no matter how much "money" you throw at it: http://en.wikipedia.org/wiki/Computational_complexity
That however does not mean that we cannot often do good approximations. For example, your GPS will find you a route. However often that route is only within a couple of percentage points of optimal, but as a human you will usually not be able to tell the difference.

 

[Dada]

Difficulty depends on the problem you are solving. Building enterprise type apps is pretty easy. Doing realtime machine vision is next level stuff...

Generally speaking it is easy( not very complex), but then again it depends on what type of enterprise application you are building, and the circumstances around it.

 

[CheekyC]

Generally speaking it is easy( not very complex), but then again it depends on what type of enterprise application you are building, and the circumstances around it.

agreed, enterprise apps can get tricky, but there is usually help on various websites. But with hardware development, machine vision and realtime DSP you are mostly in uncharted territory. DSP has lots of abstract math etc. That is where the difficulty comes in.

 

[Compton_effect]

I advise everyone to go read this. Sums it up perfectly.

http://stilldrinking.org/programming-sucks

So no, I'm not required to be able to lift objects weighing up to fifty pounds. I traded that for the opportunity to trim Satan's pubic hair while he dines out of my open skull so a few bits of the internet will continue to work for a few more days.

 

[sn3rd]

I think it comes down to the definition of development. This is a very well-defined process where a resource takes a plan and implements it. The phases prior to this, analysis, design, etc, do not form part of development.

If the prior phases are done correctly, development should not be overly challenging, and it becomes a prioritisation and tools-based task.

Having said that, most developers I have ever met tend to be responsible for much more than just development. The nature of most organisations is that money is not unlimited, so hiring separate resources for each and every task is not feasible. Time is also valuable, and additional people usually requires more time to deliver because you hit a wall when scaling a team because of the overhead of communication.

I don't consider machine learning, image processing, computer vision, etc, to be development. Rather, these fall in separate categories. For example, most of the work is done prior to even opening an editor. If you have trouble implementing a solution, your programming skill, planning or problem solving is weak.

 

[cguy]

I think it comes down to the definition of development. This is a very well-defined process where a resource takes a plan and implements it. The phases prior to this, analysis, design, etc, do not form part of development.

If the prior phases are done correctly, development should not be overly challenging, and it becomes a prioritisation and tools-based task.

Having said that, most developers I have ever met tend to be responsible for much more than just development. The nature of most organisations is that money is not unlimited, so hiring separate resources for each and every task is not feasible. Time is also valuable, and additional people usually requires more time to deliver because you hit a wall when scaling a team because of the overhead of communication.

I don't consider machine learning, image processing, computer vision, etc, to be development. Rather, these fall in separate categories. For example, most of the work is done prior to even opening an editor. If you have trouble implementing a solution, your programming skill, planning or problem solving is weak.

Most people take a broader definition (see wiki): "Their work includes researching, designing, implementing, and testing software". Using "development" as a synonym for the literal act of programming isn't a useful definition. Even "programming", hardly ever means just programming. The OP certainly took the broader sense of the word, since otherwise the post wouldn't make any sense.

 

[Hamster]

In a perfect world development is easy, in the real world though you sit with crap like:
* wanting a Bugatti but the client budget only allows for a Golf
* wanting a Bugatti but the deadline only allows for a ****ing Golf
* this leads to technical debt and "features" that just stay in the code base forever
* then there is the hardware/eco system limitations
* the client who paid for this third party app library which you MUST use riddled with memory leaks and other bugs
* PDF documents ****ing you around
* some enterprise systems are old and contain and contain a myriad of tech
* ...and then last but not least, you have a know-it-all "architect" pushing to use every latest tech

And that is what makes software development tedious and difficult.

 

[sn3rd]

Most people take a broader definition (see wiki): "Their work includes researching, designing, implementing, and testing software". Using "development" as a synonym for the literal act of programming isn't a useful definition. Even "programming", hardly ever means just programming. The OP certainly took the broader sense of the word, since otherwise the post wouldn't make any sense.

No, programming is not the same as development. As mentioned, development is about implementing the plans created in prior phases. Programming is just one small part of this.

And in the world of business, programming means programming. It doesn't mean playing around trying stuff until it works, or tuning performamce until it is satisfactory; it refers to the act of sitting and writing code.

 

[cguy]

No, programming is not the same as development. As mentioned, development is about implementing the plans created in prior phases. Programming is just one small part of this.

And in the world of business, programming means programming. It doesn't mean playing around trying stuff until it works, or tuning performamce until it is satisfactory; it refers to the act of sitting and writing code.

Now you're just using a narrow definition of programming, and passing the buck to another nebulous term "writing code". (And yes, I'm sure that "in the world of business" writing code is writing code, and tautologies are tautologies). Look, if the OP is wondering about how complex development can get, and you say that if you use a definition (which by your own statement doesn't really apply in practice) that defines it as something simple, the answer is "not very" - it's just not useful, since he is clearly asking how complex the profession of a developer can become.

 

[_kabal_]

Now you're just using a narrow definition of programming, and passing the buck to another nebulous term "writing code". (And yes, I'm sure that "in the world of business" writing code is writing code, and tautologies are tautologies). Look, if the OP is wondering about how complex development can get, and you say that if you use a definition (which by your own statement doesn't really apply in practice) that defines it as something simple, the answer is "not very" - it's just not useful, since he is clearly asking how complex the profession of a developer can become.

Next thing he is going to say how a writer just writes things on paper, that the story maker-uppers came up with.

 

[sn3rd]

You both need to learn to read (just like most developers ).

There is a difference.

A developer does not just develop, in practice. But development absolutely refers to one thing, and one thing alone (even in practice); implementing a plan set forth in prior phases. That implementation may require programming or configuration management or a number of other tasks, but it should NOT entail analysis or design. If it does, your timelines are too short or you haven't properly defined the solution yet (see analysis and design); you are setting yourself up for failure. Continuing with the examples, if I hand a developer an implementation plan, and instruct said developer to develop it, I am not asking the developer to go and design it. The developer may have been involved in the analysis and design, and may have provided valuable input (as is the developer's job, in addition to development), but the expectation is that the development phase consists of building the solution.

Using the same provided analogy, a writer does not just write, but the act of writing does not refer to coming up with the story.

The question asked was, "How difficult can development get?" And the answer is "not very".

 

[Hamster]

The question asked was, "How difficult can development get?" And the answer is "not very".

...and that is why everybody is doing it.

 

[kripstoe]

...and that is why everybody is doing it.

...but not well.

 

[Raithlin]

Such petty squabbles. So development can be challenging when solving problems that are presented in several ways: when analysing the business requirements, when designing a solution for the client, and when writing the code. At all points of the SDLC you will come up against problems needing to be solved. How difficult it is depends on many factors already listed.

It can also be simple - awfully so. Again, depending on the requirements. I guess that is the nature of the beast...

 

[cguy]

You both need to learn to read (just like most developers ).

There is a difference.

A developer does not just develop, in practice. But development absolutely refers to one thing, and one thing alone (even in practice); implementing a plan set forth in prior phases. That implementation may require programming or configuration management or a number of other tasks, but it should NOT entail analysis or design. If it does, your timelines are too short or you haven't properly defined the solution yet (see analysis and design); you are setting yourself up for failure. Continuing with the examples, if I hand a developer an implementation plan, and instruct said developer to develop it, I am not asking the developer to go and design it. The developer may have been involved in the analysis and design, and may have provided valuable input (as is the developer's job, in addition to development), but the expectation is that the development phase consists of building the solution.

Using the same provided analogy, a writer does not just write, but the act of writing does not refer to coming up with the story.

The question asked was, "How difficult can development get?" And the answer is "not very".

You truly are a master of the literal. If he asked how to become a race car driver, you would probably tell him to get in a race car and push the accelerator.

 

[Hamster]

...but not well.

Exactly.

 

[gkm]

.. The question asked was, "How difficult can development get?" And the answer is "not very".

Yes, kind of sad when people make such sweeping and narrow statements about broad subjects. And they can change the definition of any term to suit their arguments. And always assume that whatever other people is doing cannot be very hard, since they know they are a lot smarter than those people.

 

[sn3rd]

You truly are a master of the literal. If he asked how to become a race car driver, you would probably tell him to get in a race car and push the accelerator.

Once again, "racing" requires that various tasks be performed, not just driving. There is also a judgement factor involved which influences the outcome of the endeavour; placing first, second, last. This isn't something that typically happens in a corporate environment. At worst, the degree to which the developer adhered to the plan during development or the time taken to complete the development would come into question. But it is more of a binary measure, since one either meets the deadline or does not, there are usually no prizes for finishing split seconds ahead of the deadline.

 

[sn3rd]

Yes, kind of sad when people make such sweeping and narrow statements about broad subjects. And they can change the definition of any term to suit their arguments. And always assume that whatever other people is doing cannot be very hard, since they know they are a lot smarter than those people.

I'm sure you had a point you were making, but this kind of address is unlikely to receive a clear response. Perhaps you could clarify?