Insights and Technology

<div><p>I’ve been a back-end Python developer for over 7 years. Most of my professional experience has been with the Flask framework, and I’ve been quite happy with it. As far as I am concerned, the two competing options for Web frameworks in Python are <a href="https://www.djangoproject.com/">Django</a> and <a href="https://flask.palletsprojects.com/en/1.1.x/">Flask</a> (I will probably get hatemail from the Pyramid, Bottle, and other communities for this!). In my mind, I always thought that Flask was objectively superior, as it allows you more freedom of choice, is newer, and is less bloated. I was under the misconception that people chose Django simply due to its popularity. Then, after starting a new position at <a href="https://www.spiria.com/">Spiria</a>, I was handed a few projects that had been developed in Django. I found the experience eye-opening, and have developed an unexpected appreciation for the Django framework. Below, I will outline my assessment of the Django framework, from the point of view of a seasoned Flask developer. You may be reading this post wondering, “Which framework should I use?” Of course, the answer to that question is, as always... it depends!</p><h3>Project Structure</h3><p>The Django framework imposes a specific project structure. Your models, views, and routing are all placed in predictable places. The Django projects I inherited at Spiria were all initially created and developed by Rails developers and, though the codebase was sprinkled with idiosyncrasies, I still felt right at home as a Python developer.</p><p>I suspect that had the developers started the project in Flask, I would have been in for quite a lot of pain. I remember a conversation I had with a Django developer years ago, about their experience working on a project in Flask. They couldn’t understand how anyone could write a large project in Flask. After diving a bit deeper in the issue, it became clear to me that the developer’s Flask application had absolutely all of the code in a single file. Ten thousand lines in all their glory. No wonder they found the framework unmaintainable! The Flask framework imposes no inherent structure, which can be a double-edged sword. At best, you adopt and enforce a strict structure from the very beginning of your project. At worst, no structure is imposed, and you inherit a very large single-file application, giving new meaning to the concept of single-page applications! This freedom means that every single Flask application you come across will likely be structured in a different way.</p><h3>Django ORM vs SQLAlchemy</h3><p>Those of you from a Flask background will most likely have used <a href="https://www.sqlalchemy.org/">SQLAlchemy</a> for your Object Relational Mapper (ORM) needs. SQLAlchemy is an extremely powerful framework that gives you as much or as little control over your database as you need. One of the beautiful things about SQLAlchemy is that you can get close to the metal in areas where you really need fine-grained control (for example, areas with poor performance) by tapping into the <a href="https://docs.sqlalchemy.org/en/13/core/">SQLAlchemy Core</a> and writing queries in their SQL expression language. However, this flexibility does come at a cost; as they say, with great power comes great responsibility. A little lower, I’ll get into some really cool Django ORM features that are likely much harder to achieve in SQLAlchemy due to this flexibility.</p><p>In contrast, the feel of Django ORM is much more rail-like, probably due to the fact that Django ORM uses the active record implementation. In essence, this means that your Django model objects will very closely resemble the SQL rows they are derived from. This has advantages as well as disadvantages: on the plus side, the simplicity and elegance of the Django migration system (more below), without having to deal with a database session; on the minus side, not being able to automatically declare joins on relationships right on the model itself. In SQLAlchemy, this is easily done through the “Relationship Loading Techniques” of SQLAlchemy (lazy loading, eager loading, etc). To avoid the <a href="https://thenewstack.io/finding-and-fixing-django-n1-problems/">N+1 problem</a>, you will need to call <code>select_related()</code> or <code>prefetch_related()</code> everywhere you are querying for that model. This can get tedious, especially when you have two models that are almost always used together.</p><pre><code>‘‘‘Example of eagerloading taken from the SQLAlchemy docs’’’class Parent(Base): __tablename__ = ‘parent’ id = Column(Integer, primary_key=True) children = relationship("Child", lazy=‘joined’)</code></pre><p>As you can see in the example above, you can bake your joins right on your database model classes, as opposed to appending the join to the query every time you want to optimize your query. See below.</p><pre><code>parents_and_children = Parent.objects.select_related(‘children’).all()</code></pre><h3>Django Migrations vs Alembic</h3><p>Database migration is one area where Django really outperforms Flask – the built-in database migration system in Django is a pure pleasure to work with. I was particularly impressed when a teammate and I were both committing database migrations into the project at the same time: Django identified the issue immediately, and it was a trivial thing to issue a merge through the <code>Python manage.py makemigrations --merge</code> command. Technically, <a href="https://alembic.sqlalchemy.org/en/latest/">Alembic</a> has the <a href="https://alembic.sqlalchemy.org/en/latest/branches.html">branches feature</a> to deal with this issue, but it has been in “beta” for years, which does not inspire confidence for production use.</p><h3>Django Admin / Flask-Admin</h3><p>Django Admin is a major selling point for Django. Right out of the box, you get an instant <a href="https://en.wikipedia.org/wiki/Create,_read,_update_and_delete">CRUD</a> Web interface for all of your database models, tucked away behind a user login screen. The Django Admin panel is quite powerful, and a number of third-party libraries add additional functionality and quality of life enhancements.</p><p>A third-party library for Flask, the aptly named <a href="https://flask-admin.readthedocs.io/en/latest/">Flask-Admin</a>, fits the same niche as Django Admin. In keeping with the overall theme, Flask-Admin comes with much less stuff pre-configured for you. You will need to include some boiler-plate for each database model you would like added to the Flask-Admin panel.</p><p>One caveat for both Django Admin and Flask-Admin: while they can save you a lot of time up front, as your project matures and your users start requesting (or demanding) additional features in your admin panels, you will find yourself fighting these libraries to get them to do what you want. I have found these admin libraries best suited for developer use only. If you intend to open up administrative features to your user base, you are better off writing one from scratch, using server-side templating or an API with a front-end framework, like <a href="https://reactjs.org/">React</a>.</p><h3>Django REST Framework</h3><p>Speaking of APIs, I found the <a href="https://www.django-rest-framework.org/">Django REST framework</a> to be an absolute dream to work with. The class-based generic views made life so much simpler, and my codebase that much smaller. Basically, you have your views, your serializers and your permissions. Subclass the appropriate generic view (for example, a <a href="https://www.django-rest-framework.org/api-guide/generic-views/#listcreateapiview">ListCreateAPIView</a>), attach a default query, a serializer (something that defines your endpoint request/response and handles your conversion to and from JSON to Python), and any permissions that are relevant to that view. That’s it.</p><pre><code>‘‘‘Example ListCreateAPIView class’’’class BlogPostListCreate(ListCreateAPIView): permission_classes = [IsAuthenticated, BlogPostPermission] serializer_class = BlogPostSerializer queryset = BlogPost.objects.all()</code></pre><p>In contrast, building this out without the Django REST framework would look something more like this:</p><pre><code>from django.http import JsonResponsefrom django.views import Viewclass BlogPostListCreateView(View): def get(self, request): blog_posts = BlogPost.objects.all() blog_post_dicts = [] for blog_post in blog_posts: blog_post_dicts.append({ ‘id’: blog_post.id, ‘title’: blog_post.title, ‘body’: blog_post.body }) return JsonResponse({‘blog_post_dicts’: blog_post_dicts}) def post(self, request): new_blog_post = BlogPost.objects.create(title=request.POST[‘title’], body=request.POST[‘body’]) return JsonResponse({‘message’: ‘Blog Post Created’})</code></pre><p>The example above does not include permission checking, validation, or pagination, which you would get for free with the Django REST framework.</p><p>An added bonus is all of the additional libraries that were developed around the Django REST framework, that make your life as a developer so much easier. Two notable libraries I’ve used with the Django REST framework are <a href="https://pypi.org/project/django-rest-framework-camel-case/">Django-Rest-Framework-Camel-Case</a> and <a href="https://drf-yasg.readthedocs.io/en/stable/">drf-yasg</a>. The former ensures your API endpoints can accept both CamelCase and snake_case keys on the request and uses CamelCase for the response, all the while maintaining snake_case variable names throughout your Python code. This allows your front-end developers to use the variable naming conventions that they are used to, and allows your back-end Python developers to use snake_case and conform to <a href="https://www.Python.org/dev/peps/pep-0008/">PEP8</a>.The latter library, drf-yasg, works well with the former, and helps to automatically generate <a href="https://swagger.io/">swagger</a> documentation by parsing out your generic views and their serializers. Nothing beats automatically-generated documentation!</p><p>To be fair to Flask, a number of third-party libraries likely do a lot of what the Django REST Framework does with Django. However, for some reason, I never reached for one of those tools during my time developing in Flask, which warrants further investigation in itself.</p><h3>Conclusion</h3><p>I’m grateful for the opportunity of trying out Django in a professional setting. Prior to joining Spiria, I was a die-hard Flask developer and did not look at the other side of the Python Web framework coin. Flask is an amazing framework, and the freedom it provides allows you to do so much. However, this freedom demands a lot of discipline, and a wrong decision at the architecture stage can cost you dearly as your project matures. Moving to Django opened up my eyes to a lot of quality of life enhancements that I didn’t know I was missing out on in Flask. Of course, there were a few things I missed from Flask (automatic joined loads, for example), but, for the projects I’ve worked on so far, I really did feel like Django was the right choice.</p><p>My recommendation would be to reach for Flask when making a quick prototype, or when producing something so customized that Django’s rails would be too restrictive. On the other hand, reach for Django for those projects that can benefit from having that much structure, especially when working with a large team. There is definitely a place for both Flask and Django in the world, and the Python community as a whole is better off for having these choices. Thank you to everyone who contributed to both projects and all the open-source satellite projects that help make both frameworks a dream to work with.</p></div>

<div><p>We used JMeter to do some performance testing for one of our clients, targeting 400 users concurrently. We encountered a few challenges initially, which we overcame by optimizing the code, by using the right kind of samplers and their properties, the right kind of thread groups to achieve the desired level of concurrency, and of course by using the listeners only for debugging, and not for actual load testing.</p><p>Further challenges came up when the target concurrency was bumped up to 1,000 users at a time, since JMeter could not handle that big a volume. For example, it started fading out while putting the same amount of load on AWS servers, and load test results were showing JMeter errors rather than actual server failures. Serious configuration changes were required to scale up JMeter to a level where it could generate the desired load and make AWS servers fail.</p><p>Here are our top tips to help scale up JMeter to put AWS servers through their paces:</p><h2>Distributed Testing</h2><p>Distributed testing is your first and best strategy to generate enough load to take an auto-scalable, perfectly load-balanced AWS server to the breaking point. Running JMeter locally limits the number of concurrent users that can run on a given system at a given time, meaning you rely squarely on the CPU and memory of the system the load test is running on.</p><p>Using master slave mode, running the test on two or more servers spreads the load across systems, which helps scale up JMeter to the size and space of all the master-slave machines combined. Distributing the machines not only spreads the load to generate an enormous number of threads every second, but it also distributes test data CSV files and huge test result files for more scalability. The online community attests to the running of 250 to 500 concurrent threads per server. You can add machines to the distributed system based on the overall load generation needs.</p><h2>Increasing the number of TCP connection ports</h2><p>A typical Windows machine will only make outbound TCP/IP connections using ports 1024-5000, and takes up to 4 minutes to recycle them. Running load tests requires a lot of connections in a short amount of time, quickly saturating that port range. When we attempted to target 400 concurrent users to load test the AWS services, the JMeter machine running on Windows OS could not even reach this target concurrency.</p><p>To increase the maximum number of ephemeral ports, follow these steps:</p><ol> <li>On each JVM, start Registry Editor.</li> <li>Locate the following subkey in the registry, and then click Parameters:<br> HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters.</li> <li>Right-click on Parameters.</li> <li>Create a new DWORD value with name MaxUserPort.</li> <li>Right-click on MaxUserPort.</li> <li>Enter 65534 for the Value Data field.</li> <li>Under the radio button selection for Base, select the Decimal radio button.</li> <li>Click OK. Close the registry.</li> <li>Restart each JMeter load agent machine.</li></ol><h2>Increasing heap size</h2><p>After increasing the number of TCP connections using ports, we scaled up JMeter to the desired concurrency level. Soon, however, JMeter machines started producing “Out of memory” errors. It was essential to run load tests using GUI mode, as manual intervention was required to start and stop multiple thread groups while the application kept changing its state from inactive, to active, to closed. Even though only Aggregate Listener was used with the least amount of configuration to save space, we kept getting errors due to memory issues.</p><p>JVM can be fine-tuned to increase the allocated RAM. When a Java program like JMeter requires a fair amount of memory, RAM is commonly reallocated.</p><p>JMeter uses up to 512MB of RAM by default, as stated in jmeter.sh or jmeter.bat launch scripts. We increased the heap size to 4G:</p><pre><code>set HEAP=-Xms1024m -Xmx4096m -XX:MaxMetaspaceSize=1024m</code></pre><p>However, running JMeter after increasing the heap size was producing the following error:</p><blockquote><p>Invalid initial heap size: -Xms4g<br>The specified size exceeds the maximum representable size.<br>Error: Could not create the Java Virtual Machine.<br>Error: A fatal exception has occurred. Program will exit.<br>errorlevel=1</p></blockquote><p>To address this error, we had to upgrade Java (JDK specifically) and JMeter to the latest versions. Indeed, the latest JDK contains a server optimized compiler that is better at optimizing Java code execution, thus JMeter Execution. After downloading and installing the latest JDK version, we restarted the JMeter server, and voila!, no more errors cropped up.</p><p>We were able to successfully run the load tests with 1,000 concurrent users, spiking up to 3,000.</p><h3>References</h3><ul> <li>Base Logic: “<a href="https://www.baselogic.com/2011/11/23/solved-java-net-bindexception-address-use-connect-issue-windows/">Solved ‘java.net.BindException: Address already in use: connect’ issue on Windows.</a>”</li> <li>OctoPerf: “<a href="https://octoperf.com/blog/2017/10/17/solve-jmeter-out-of-memory-issues/">JMeter out of memory: 12 actionable fixes.</a>”</li></ul></div>

<div><h2>First comes an email with an invitation.</h2><p>Shortly after you successfully submit your resume and cover letter (see our <a href="https://www.spiria.com/en/blog/working-environment/spirias-recipe-for-a-tasty-it-resume/">recipe for a tasty IT resume</a>), you should receive a system-generated acknowledgment of the receipt of your application. Within 2 to 3 business days, our recruiting team will study your resume and, if your skills match our needs, we’ll send you an email invitation to a phone call. We won’t call you without your express consent, so please monitor your spam folder in case our invitation lands there.</p><h2>Then, you choose the time and day of our call.</h2><p>After you receive our phone invitation, review the job posting you applied for and the CV you submitted, to make sure we’re all on the same page. While you’re at it, peruse <a href="https://www.spiria.com">Spiria’s website</a> to gain a better understanding of our company, and check the location of our offices to make sure it meets your needs. Finally, give us the day and time that suit you best. Don’t be afraid to schedule a call during lunchtime, after your classes, or before or after work. Our recruiting team will do their best to accommodate you; we don’t want to cause any inconvenience or stress. And, of course, prepare your questions for us.</p><h2>Finally, the talk.</h2><p>Our phone interview questions are simple and straightforward.</p><p>We’ll ask you:</p><ul> <li>about your current employment status;</li> <li>how you learnt about the position at Spiria;</li> <li>why you’d like to work with us;</li> <li>why you’re looking for a change, as the case may be;</li> <li>about your experience and skills;</li> <li>about your day-to-day life on the job;</li> <li>about the technologies you know best;</li> <li>and about the languages and technologies you’d like to learn in the future.</li></ul><p>We might test your language skills and, if you’re bilingual, we might switch between French and English.</p><p>We’ll also ask you about your salary expectations. Don’t be afraid of this question; just do some research beforehand and give us a bracket you would feel comfortable with. Spiria also has its own brackets for each position within a salary grid comprised of salary bands. Each band has a corresponding salary range with a starting rate, several midpoints, and a maximum. This grid helps to ensure that all our employees are paid fairly based on their skills, their progress in their position, their colleagues, new hires and, of course, the Canadian market.</p><p>At the end of the call, we’ll ask you about your availability for an in-person interview, so check your schedule ahead of the call for the next few days. Of course, we’ll be more than happy to answer all your questions. Spiria’s phone interviews never last longer than 25-30 minutes. Our goal is to get to know you better and to introduce ourselves. Our Talent Acquisition team is genuinely interested in potentially hiring you, and will uphold your interests to hiring managers.</p><h2>Decision time.</h2><p>Usually, you’ll hear from us within a week. If you don’t receive an answer within that timeframe, don’t hesitate to send us a follow-up email. If our hiring managers are interested in you, our Talent Acquisition team will contact you to schedule an in-person or Zoom meeting. You’ll also receive a short, 10-minute personality test, as well as a technical test, which can take up to one hour to complete.</p><p>If you want to know about the next steps in Spiria’s hiring process, how to prepare for an interview and what tricks to expect from our hiring managers, stay tuned, follow Spiria, and we’ll tell you everything in our next posts.</p><h3>See also:</h3><ul> <li><a href="https://www.spiria.com/en/blog/working-environment/preparing-for-a-successful-job-interview-with-spiria">Preparing for a Successful Job Interview with Spiria</a>, by Amanjot Kaur.</li> <li><a href="https://www.spiria.com/en/blog/working-environment/spirias-recipe-for-a-tasty-it-resume">Spiria’s Recipe for a Tasty IT Resume</a>, by Lidiia Meleshchenko.</li></ul></div>

<div><h2>Do your research on Spiria</h2><p>Figure out how we define ourselves as a company. Educate yourself on our mission. Explore the types of services we offer.</p><p>Prepare some questions on the things you don’t understand or would like to know more about. Check our website for answers to the following:</p><p>1. Where are the company’s offices located?</p><p>2. Does Spiria have a single, national, or global presence?</p><p>3. What specific qualities does Spiria look for in employees?</p><p>Make a mental note of the areas that are particularly interesting and motivating for you.</p><p>“Why do you want to work specifically for Spiria?” is a question that usually gets asked in interviews.</p><p>You can find useful information on Spiria outside our website. For example, Glassdoor has employee reviews and valuable information about the interview process.</p><h2>Research your interviewer</h2><p>If you know the name of your interviewer, check out their profile on LinkedIn. Look for common interests, which will create an opportunity to build a rapport. You can also get a sense of their level of expertise and experience in their current role in the company.</p><h2>Prepare for predictable interview questions</h2><p>Any interviewee should have ready answers to a number of standard interview questions about themselves, their career goals, their motivations and their interest for their future role at the company.</p><p>Some common questions include:</p><p><i>Tell us about yourself. </i></p><p><i>Please walk us through your experience so far.</i></p><p><i>Why do you want to work for Spiria?</i></p><p><i>Is this office location easy to access for you on a daily basis?</i></p><p>There may be a chance for some technical interview questions as well! Be prepared for <b>whiteboard trouble-shooting or coding</b>.</p><p>There may also be questions on your experience in client relations:</p><p><i>How do you handle an unreasonable <b>client</b>?</i></p><p><i>What do you consider to be good <b>customer</b> service?</i></p><p><i>What would you do if a <b>client</b> had a problem that you weren’t sure how to resolve?</i></p><p>To answer these questions, we suggest using the <b>STAR </b>method:</p><p><b>Situation</b>: What was the situation?</p><p><b>Task</b>: What were his/her tasks, responsibilities, role?</p><p><b>Action</b>: What action did he/she take?</p><p><b>Result</b>: What was the result?</p><h2>Preparing for a Zoom interview</h2><p>Video interviews are becoming increasingly common.Thanks to improvements in technology, most job searchers can do a Skype, Zoom, or Google Hangouts interview quite easily. Plus, a video interview gives you the face-to-face aspect missing from a phone interview—without having to take time off work (or travel across the country!) to meet in person. Spiria typically uses the Zoom platform.</p><p>Here’s how you can prepare:</p><ul> <li>Choose a spot that lets you control your surroundings.</li> <li>Take advantage of natural light from a window.</li> <li>Ensure the camera is close to eye level.</li> <li>Keep a copy of your resume, a notepad and pen, and a glass of water by your side.</li> <li>Dress professionally from head to toe.</li> <li>Test your audio and internet connection beforehand.</li> <li>Ensure your background is simple, with no distractions.</li> <li>Make sure your head and shoulders are visible.</li> <li>Headphones help prevent echo.</li> <li>Smile and nod to show you are listening.</li></ul><h2>The day before the interview</h2><p>Before interview day, ask your Spiria contact where you should park and where you should meet them in the building.</p><p>If you’re unsure about Spiria’s dress code, ask! In general, it’s better to be overdressed than underdressed, but it’s also wise to just ask instead of assuming. Lay out your outfit the night before and do any ironing or steaming to save you time and stress in the morning.</p><p>Essential things to bring to the interview: several copies of your resume, your portfolio, several copies of your business card (if you have one), a notepad and pen, and a list of professional <a href="https://www.inhersight.com/blog/interview/professional-references">references, just in case</a>. Basically, bring anything and everything that can support your expertise and back up your words.</p><h2>The morning of the interview</h2><ul> <li>Give yourself plenty of time to get ready so you aren’t rushing around. Also give yourself an extra 15–30 minutes to get to your interview location in case of traffic or an accident.</li> <li>Do whatever relaxes you the most, whether it’s 15 minutes of yoga or slowly sipping your coffee.</li> <li><a href="https://www.inhersight.com/blog/data-research/5-best-pre-interview-breakfasts-according-inhersig">Eat a hearty breakfast</a> (it’ll boost that brain power!).</li> <li>Recite some affirmations and tell yourself you’ve got this. Confidence is everything during interviews.</li></ul><h2>After the interview</h2><p>Reach out and let us know how your experience was throughout the entire process. We love to ensure our candidate experience is the best out there! We are always looking to improve and grow.</p><p>You are now ready to impress your interviewer with your knowledge of the job and company.</p><p>Good luck! ☺</p></div>

<div><p>After all, it is your first opportunity to make an impression, and it can highly influence your success at securing a job. That’s why you want it to look good and whet the appetite. In this time of pandemic, when many of you have lost jobs, were laid off, or thrust in circumstances you were not prepared for, the Spiria Talent Acquisition Team would like to help you to spice up your resume and cook it up to our hiring managers’ tastes.</p><p>Below, you will find our recipe for a well done, professional IT resume.</p><p>1. <b>Make it short and sweet.</b> Ideally, your resume should be 1 to 2 pages, or one page for every 10 years of work experience. In this competitive world, we receive dozens of resumes every week. Make sure yours makes us notice how great you are in one glance.</p><p>2. <b>Are you bilingual? If so, state it right off the top</b>. We have both French- and English-speaking customers and, quite often, they are looking for bilingual specialists.</p><p>3. <b>Location. </b>Indicate your city of residence, and whether you are willing and able to work remotely. With the COVID-19 situation, more and more companies are having their employees work remotely.</p><p>4. <b>Tell us who you are. </b>Right after your name and contact details, add one or two sentences summarizing who you are and what your superpower is. Indicate your top three technologies (for example “Senior back-end developer, specializing in Python, 9 years of experience”; “Intermediate full-stack dev, 5 years of experience with JavaScript, C#, .Net”)</p><p>5. <b>Personal information is sensitive.</b> We don’t need to know your date of birth, marital status, number/age of kids, gender, religion, race, colour, or national origin; even your photo is TMI.</p><p>6. <b>Previous professional experience</b>. For each employer, state the name of the company, its location, your role and dates of employment. Summarize each of your experiences with 3 or 4 bullet points.</p><p>7. <b>Technologies.</b> Indicate only those technologies that you are experienced and comfortable in working with. Don’t include all those that were quickly covered in class. And you can omit the basic technologies that go without saying, for example HTML for a web developer. ;)</p><p>8. <b>Do you have</b> <b>customer relations experience</b>? Bonus! Spiria is a custom software company where employees have the opportunity to communicate with clients. So please, please mention that asset of yours.</p><p>9. <b>Explain gaps in your career path.</b> For example, you graduated in 2010 but only got your first job in 2015? Don’t let us fill in the blanks. Explain whether you were working as a freelancer, launched your business, travelled around the world or worked in a grocery store (customer experience is an asset, remember?).</p><p>10. <b>Education</b>. Do include relevant school achievements, participation in hackathons and other sprint-like events. But please, don’t include your university transcripts, just the outcome.</p><p>11. <b>Personal achievements.</b> If you have achieved something special, or if you are involved in volunteer activities, let us know, OK? We are interested in your personality and values.</p><p>12. <b>“References available upon request.”</b> Not necessary. It adds nothing to your resume, and detracts from all the greatness above. Don’t worry, before hiring you, we’ll ask for your references.</p><p>13. <b>PDF, not Word</b>. Don’t send your resume as a Word Document. All your careful formatting will just get messed up when we open the file on our platform.</p><p>What happens next? If we like your resume, we will send you an email to invite you for a phone interview. Therefore, after <a href="https://www.spiria.com/en/career/">submitting your resume</a>, please make sure to check your emails regularly, including your spam folder; as an unknown sender, we might end up there. To find out the following steps in our hiring process, stay tuned and follow Spiria. We will tell you more in our next article.</p><p>Good luck to you all. And hopefully we will meet in person soon! :)</p></div>

<div><p>All these systems have to be initially thought up and developed, then continuously maintained throughout their lifecycle. And we use such systems every single day, whether we realize it or not. But have you ever given any thought to the work of the programmers behind the tools that we couldn’t function without?</p><p>As a developer specializing in Web applications, I’d like to give you a peek at my day-to-day work, tasks and challenges. Of course, every <a href="https://www.spiria.com/en/services/purpose-built-development/custom-software-development/">software development project</a> is unique, and every team has its own way of doing things, but this overview is still representative of the general approach taken for most software development projects.</p><p>But let’s start with clearing up a concept first: the all-encompassing word “development”, which covers the general process of causing a piece of software to be born, isn’t just about developers. In fact, development calls upon many different types of knowledge and professionals that have nothing to do with writing code.</p><h2>The software development cycle</h2><p>In the modern software development cycle, called “Agile”, an application is in continuous development:</p><p><img src="https://mirror.spiria.com/site/assets/files/4639/sotware_cycle.png" style="width: 50%; border: none;" alt="Development cycle." title="Development cycle."></p><ol> <li><b>Analysis</b> — Needs analysis: new features required, bugs to fix, etc.</li> <li><b>Design</b> — Development of mock-ups, software architecture, etc.</li> <li><b>Implementation</b> — Development per se (“coding”).</li> <li><b>Tests</b> — Quality Assurance (QA) to confirm the system is working as expected and to find bugs that may have gone unnoticed during development.</li> <li><b>Deployment</b> — Launch of product.</li></ol><p>Best practices require the development process to go through each of these stages in regular cycles, in order to obtain regular updates during production. Each cycle, called “sprint”, typically lasts anywhere from one to four weeks, depending on the nature of the project. However, in case of an emergency, for example when a “<a href="">hotfix</a>” is required, an entire cycle can be completed in as little as a day.</p><p>The other thing to remember is that not all cycles produce changes that are visible to the end-users of the application. For example, a cycle could deal just with underlying analysis or background architecture changes to lay the groundwork for future features, or to stabilize the platform. Also, any given feature can be split up in several smaller, simpler tasks to spread the load between several developers.</p><p>Sometimes, depending on how the teams are organized within the company, developers are called upon to take on tasks at every phase of the cycle. A good developer is therefore versatile and able to quickly adapt to any given situation.</p><p>While most companies have a dedicated QA team to test applications and catch bugs before the launch of any new features, developers still have to test their code themselves during implementation to fix as many bugs as possible, as early as possible, in the production process.</p><p>Also, the different phases of the development cycle are not always sequential. For example, the analysis and design phases of the next cycle can be performed simultaneously with other phases of the current cycle.</p><p>The analysis phase usually involves the following tasks:</p><ol> <li>Needs analysis: Identification and prioritization of bugs and new features for the next cycle.</li> <li>Documentation: Documenting technical requirements to meet the identified needs. Splitting up of complex features in several simpler sub-tasks.</li> <li>Estimation: Estimating the complexity of each of the identified tasks, and time required to complete them.</li> <li>Validation: Planning and validating the contents of the next development cycle.</li></ol><p>The needs analysis phase is initiated in close collaboration with the client, represented by the “Product Owner”; the development and Quality Assurance teams usually get involved at the final stages of estimation and validation.</p><h2>The coder’s work</h2><p>As for actual development work, developers usually undertake the following work for each task:</p><ol> <li>Implementation of a task in the code (adding the feature in the code or adjusting existing code to fix a bug): <ul> <li>Research and documentation (using an external software library, finding an existing solution to fix a bug, exploring technological alternatives, etc.)</li> <li>Adding <a href="https://en.wikipedia.org/wiki/Unit_testing">unit tests</a>, functional tests or integration tests, depending on the project.</li> <li>Testing new features in a dedicated development environment.</li> </ul> </li> <li>Opening up <a href="https://en.wikipedia.org/wiki/Code_review">code review</a> to other developers: <ul> <li>Adjustments and corrections based on the code review feedback (if applicable).</li> </ul> </li> <li>Deploying new features in the dedicated QA environment: <ul> <li>Adjustments and corrections for bugs found by the Quality Assurance team (if applicable).</li> </ul> </li></ol><p>Once all the tasks are completed and collated, the completed “sprint” can be deployed to production.</p><p>Finally, it’s important to know that all of the various phases of the development cycle are not stand-alone, but rather interrelated.</p><p>For example, after the implementation of a new feature, if a bug is found by the <a href="https://www.spiria.com/en/services/purpose-built-development/quality-assurance-test-automation/">Quality Assurance</a> team, the developer responsible for the task will fix it by going back to the implementation stage.</p><p>On the other hand, if a bug is found in actual production during a cycle, it is prioritized and analyzed. Depending on its seriousness, it can be corrected as a “hotfix”, or inserted in the cycle currently underway, potentially replacing another feature in the next deployment, or fixed in a future cycle.</p><h2>Collaboration and communication</h2><p>Collaboration and communication between the various teams is the secret to developing a robust, quality application efficiently. Developers must be in constant contact with each other and with the other members of the team, as needed, throughout the cycle.</p><p>Ideally, communication takes place directly and in person, but depending on the size of the company and the location of the client, sometimes, teams are spread out in different cities, countries, or even continents. Meetings and discussions that take place virtually are no less important than in-person ones. Nowadays, with many companies allowing their employees to work remotely, the quality and frequency of communications within the team is all the more important.</p><p>A typical day for a developer would be made up of the following tasks, not necessarily in the following order:</p><ul> <li>Check in with the rest of the team (for example, a “scrum” meeting).</li> <li>Review code for features completed by other developers.</li> <li>Adjustment of completed features following code review by other developers or feedback from the QA team.</li> <li>Development of new features (implementation + test).</li> <li>Contribute to estimating future features (generally once per “sprint” or cycle, depending on requirements).</li></ul><p>Of course, all of this will vary depending on the project and the role of each developer within the team.</p><p>In conclusion, software development is a complex process that is constantly evolving. Aside from developers, it involves many different players that are just as important as the developers. Next time you use your favorite application, give a thought to the amount of work and effort that was put in by all the various development teams to make it happen.</p></div>

<div><h2><a href="https://www.merriam-webster.com/dictionary/TL;DR">TL;DR</a></h2><p><b>Check your ego at the door</b><br>Your ego is only going to slow you down in the process of becoming a better programmer. Leave it behind.</p><p><b>Teamwork is essential</b><br>You won’t go far without the team. Put aside your own interests and look at the big picture.</p><p><b>Understand that everybody has different needs</b><br>Not everybody works and thinks the way you do. It doesn’t mean that they’re wrong, it just means that they have different needs; and what you need is to accept it.</p><p><b>Leave your comfort zone</b><br>Avoid being comfortable, and explore the vast stretch of the tech world. Learn something completely unrelated to your work; you never know when it’s going to come in handy.</p><p><b>Don’t burn yourself out</b><br>It’s not a sprint, but a marathon. Take some time for yourself and enjoy life outside of work.</p><h2>Check your ego at the door</h2><p>The sheer amount of knowledge required to bring a project from idea to fruition is just humongous. You need relevant skills in UI/UX, back-end, front-end, database, devops, architecture, security, and the list goes on. Should you jump on Pluralsight and try to learn it inside out? If you want to spend years doing nothing else, go ahead. Or, you could try to learn from your peers who spent many years perfecting their craft.</p><p>If you have a large ego, not only are you not going to listen and learn, but your teammates won’t be inclined to teach you anyway. You will take every correction or suggestion for improvement as a personal affront. You will end up wasting a lot of effort trying to protect something useless: your ego.</p><p>This one hit me hard. I thought I knew it all until I realized that… I didn’t. What knowledge I had was just a drop in the ocean. I realised how big and stupid my ego was when I started working with people that were far more knowledgeable than me. Letting go of your ego is easier said than done, and it’s not achieved overnight, but the benefits make it worthwhile. I recommend the book <i>Ego is the Enemy</i>, by Ryan Holiday. This excellent book is an easy read, and it provides insights on why your ego is of no use to you and why you should divorce yourself from it.</p><h2>Teamwork is essential</h2><p>Behind every great software is a great team that built it. I didn’t realize just how important teamwork was until I was thrust in a real environment, with a real project, and a real team. It’s a different dynamic from what I was used to, when my own interest came before the team’s interest.</p><p>Communicate often, ask questions when you’re not sure about something, flag issues or problems when they arise. Everybody needs to do their part in the interest of project coordination. Don’t be uncommunicative for days on end. Your decisions need to be made in the best interest of the team instead of yourself, and you need to work with the team and not against it. Remember, nobody likes a “hero” programmer who thinks he can do everybody’s work.</p><p>I am more of a lone wolf, a thinker before a speaker, an introvert. For me, communicating and being there for the team was never an issue because I didn’t need to. In my previous jobs and in school, I could just plug in my earbuds and work the way I wanted, at the pace I wanted, with the tools I wanted. Eventually, though, I had to adjust my behavior and habits, because as fun as it is to have complete freedom, achieving something great and meaningful requires more than one person. Alone, you go faster, together, you go further.</p><h2>Understand that everybody has different needs</h2><p>You will work with a lot of people throughout your career. Since every person is unique, your working relationship with each and every person should also be unique. That means that you should try to figure out any given person’s needs so that you can better understand why they think and act the way they do.</p><p>That doesn’t mean that you need a Ph.D. in psychology. Instead, read <i>Interpersonal Styles</i>, by Larry Wilson. It gives great insights on the concept of “core persona” in layman’s terms. In short, you have:</p><ul> <li>The analytical: data-oriented, needs clarity and objective answers.</li> <li>The driver: goal-oriented, needs to move fast and efficiently.</li> <li>The expressive: feeling-oriented, needs freedom and stimulating tasks.</li> <li>The amiable: social-oriented, needs a positive atmosphere and social interactions.</li></ul><p>The goal is not to try to please everyone all the time, but rather to show empathy when somebody might not feel impelled by their work. It’s a great tool to help you better understand your teammates and adjust your working relationship with them accordingly.</p><p>For the longest time, I didn’t understand people who constantly needed answers regarding their work. I thought, “When you don’t know, just make a decision and run with it. How hard can it be?” I always felt like they needed spoon-feeding, when in fact, they just needed clarity. They are analytical, and that’s just their nature. I worked faster than them, but my work would need revisions, while theirs would be approved upon review. I needed speed, while they needed clarity. There is no right or wrong, just different needs.</p><h2>Leave your comfort zone</h2><p>Don’t be afraid to try something completely new. There are so many different sectors, languages, frameworks, workflows, architectures, paradigms, and the list goes on. Even if it’s not directly related to your line of work, you should explore it. Chances are that what you learned by leaving home will benefit your work in one way or another.</p><p>Take some time each week to explore, even if it’s just a couple of hours. There is something euphoria-inducing when you learn something out of sheer curiosity. You have more time to analyze, question and dig than when you have to learn the same stuff in a hurry for a project.</p><p>No knowledge is ever wasted, and you should take full advantage of your youth to make good use of the time you have left on the clock. Time is your greatest asset. Make it work for you.</p><h2>Don’t burn yourself out</h2><p>It’s perfectly natural, when starting out, to want to prove yourself by working like a madman. You’re young, you’re starting your career and you want everyone to know that you’re not going to be dead weight. But how long can you keep up the pace before you’re out of fuel? A few months, if you’re lucky? After that, you’ll feel the same way you do after a sugar crash: out of motivation and concentration.</p><p>If you seek respect, this is not the way to earn it. More senior workers have seen it before, and they know that the crazy amount of story points you’re racking up now is not going to last. Don’t squander your nitrous at the start of the race; keep it for when you’re really going to need it. There are going to be times when you do need to double down, and it’s going to be a nightmare if you’ve spent all your fuel.</p><p>I am guilty of doing it. I thought that the more user stories I put out, the more valuable I would become. I tried to be the hero nobody asked for. It was short-lived. I felt great at first. I received a couple of pats on the back, a couple of mentions in meetings. And then the months passed. My motivation tanked. I had little patience and less concentration: a deadly combo. All I wanted was a break, but I couldn’t have one. I had to take many one-day breaks to stay sane.</p><p>Moral of the story: take time for yourself, exercise, get back to the hobbies you put aside because you were too busy with your crazy schedule. Travel and read. Staying happy outside of work is the best way to be happy at work.</p></div>

<div><p>The question we will explore is: <i><b>How fast can a dictionary access get?</b></i></p><p>Yes, it’s a bit weird as code golf goes, but bear with me, we’ll get to the green with this one. Maybe not in the usual manner... but we’ll get there, together, as a team! Just kidding, no teamwork required. I’ll do all the swinging and trudging, and you can just follow along in the cart.</p><p>All the code we will discuss is <a href="https://github.com/pierrebai/FastDict">available on GitHub</a>.</p><p>(Note that there are multiple branches: “master” for the starting point and “fast-dict” for the final result.)</p><h2>The Rules</h2><p>As you’ll see, by the end of this, the rules of golfing will be bent, and we’ll make some up along the way. And I’ll be setting down ground rules, for example what the dictionary must basically look like and the functionality it must provide.</p><p>First, I’ll provide a high-level description of the functionality, then a short C++ declaration of what the interface will look like. For a full view of the C++ API, refer to the “master” branch on GitHub. Now, on to the description!</p><h3>High-level Description</h3><table style="width:100%"> <tbody> <tr> <th>C++ Class</th> <th>Purpose</th> <th>API</th> </tr> <tr> <td style="vertical-align: top;"><code>element</code></td> <td style="vertical-align: top;">The dictionary item. Contains a value of any type. For example: an integer, a double, some text, or another dictionary.</td> <td style="vertical-align: top;">Setting values, reading the value, resetting the element, comparing elements, etc.</td> </tr> <tr> <td style="vertical-align: top;"><code>name</code></td> <td style="vertical-align: top;">A label-like type used to access elements in the dictionary.</td> <td style="vertical-align: top;">Constructing new names, comparing names.</td> </tr> <tr> <td style="vertical-align: top;"><code>dict</code></td> <td style="vertical-align: top;">The dictionary, indexed by name, holding elements.</td> <td style="vertical-align: top;">Construction, copying, merging, adding, removing and accessing its elements, over its elements.</td> </tr> </tbody></table><h3>C++ Code</h3><p>In this article, I will only give the shortest of descriptions; the full code can be found on GitHub. Since the main thing that interests us is the dictionary access, that’s all I’ll show right now:</p><pre><code> // Element retrieval. element &amp; operator [](const name &amp;); const element &amp; operator [](const name &amp;) const;</code></pre><p>Very simple, very standard. The look-up function receives a <code>name</code> and returns an <code>element</code>. To help understand what <code>names</code> are, let’s see how they are created:</p><pre><code> // You need to derive from name to create a concrete name. // All instances of a given concrete name are equal. struct name { // Invalid-name, default constructor. name() : _name(nullptr) {} protected: // The text passed for the name by its sub-classes must be static // so that each address is unique. name(strptr n) : _name(n) {} };</code></pre><p>While <code>name</code> has a public default constructor, its other constructor is protected. Why is that? You will see the underlying reason later on… Does that mean that all instances have to come from sub-classes? Yes! Every <code>name</code> will be an instance of a sub-class of <code>name</code>! But which sub-class? Gee... <b>all of them</b>, of course!</p><p>In fact, if you look in the GitHub repo, you will see that I provide a handy <code>voc.h</code> header. This header declares… a vocabulary of names. As was hinted in the <code>name</code> class declaration, to each <code>name</code> its sub-class… to each sub-class its <code>name</code>! The file goes a bit like this:</p><pre><code> namespace voc { #define MAKE_NAME(n) struct n ## _n : dak::name { n ## _n() : name(L ## #n) {} }; MAKE_NAME(apple); MAKE_NAME(person); // etc… const apple_n apple; const person_n person; }</code></pre><p>The <code>element</code> class itself is a simple class that can hold a value of any common type. There is nothing special about it. Now would be a good time to think of just how fast you could make such a dictionary interface. You can look at the “master” branch on GitHub as a starting point. Will it be N log N? N?, log N? Constant time? Faster? Wait, faster than constant time? What does that even mean?</p><h2>The Bends</h2><p>Now that we’ve set down the ground rules, it’s time to bend them to our advantage. Of course, like any self-respecting cheat, we carefully thought out the rules to give the (club)house an edge.</p><p>For example, there is a good reason why the <code>name</code> class is designed the way it is. You see, having different types for different names means that we can subvert the C++ type system to our advantage. More specifically, we can subvert the function overloading mechanism!</p><p>Remember the <code>element</code> access functions? What would happen if they were overloaded like this:</p><pre><code> // Element retrieval. element &amp; operator [](const name &amp;); const element &amp; operator [](const name &amp;) const; // Overload!!! inline element&amp; operator [](const voc::rock_n&amp;) inline const element&amp; operator [](const voc::rock_n&amp;) const</code></pre><p>It means we can return the <code>element</code> without having to look it up! Shameless cheating! But how can the <code>element</code> be found if the <code>voc::rock</code> is accessed through the version taking a simple <code>name</code> and not a <code>voc::rock</code>? How can the <code>dict</code> elements be iterated over during normal iteration? Easy! We create proxy elements in the normal dictionary map, each proxy defers all its behavior to the direct-access copy. Basically, we add a few functions to the <code>element</code> class to record if it is a proxy. We also add a function to the <code>dict</code> class to record each proxy <code>element</code> and the direct-access <code>element</code> it refers to.</p><pre><code> struct dict { protected: std::map<name element=""> _elements; // Sub-classes call this during construction // to add the permanent proxy elements. void add_permanent_proxy(const name&amp; n, element &amp;); }; struct element { bool is_proxy() const; bool is_permanent() const; };</name></code></pre><p>The result is that we can access the elements of our choice at compile-time! You simply sub-class the <code>dict</code> class and add the proxy elements that will be accessible under the names of your choice. The resulting class acts just like a dict and can be used everywhere a <code>dict</code> can be used, but if you know the true type of the <code>dict</code> and the true name you want to access, you get compile-time access thanks to function inlining and overloading.</p><h2>The Twist</h2><p>We in the madness business are not satisfied with such fairway-variety trickery. This subversion doesn’t go far enough. We want more speed! We do have compile-time access to our element, but what we want is compile-time access to the <b>value</b> contained in the element. Is that even possible? Why, yes, it is!</p><p>The sleight-of-hand we will be using is sub-classing the <code>element</code> class where the value resides. If we know in advance the type of value we want to keep under a <code>name</code>, we can force it to always have that type known at compiling time. Knowing the type of value we want to keep under a specific <code>name</code> is not unusual, it’s even typical! That’s how we design types, schema and databases, after all.</p><p>So here is a typical example of such sub-classing. (See the “fast-dict” branch on GitHub for all the variation provided.)</p><pre><code> struct eint64 : element { operator int64&amp;() { return _i; } // etc... };</code></pre><p>As we can see, it can inline the access to the true value contained in the <code>element</code>. Now our <code>dict</code> sub-class can return such an <code>eint64</code> in its overloaded <code>element</code>-access function, and offer full compile-time access to the direct value, like this:</p><pre><code> inline eint64&amp; operator [](const voc::rock_n&amp;) { return _rock; } inline const eint64&amp; operator [](const voc::rock_n&amp;) const { return _rock; }</code></pre><p>To support the <code>element</code> sub-classes, an additional function is added to <code>element</code> to let it know that the <code>value</code> type is now fixed:</p><pre><code> bool is_fixed() const { return _fixed == fixedtype::fixed; }</code></pre><h2>The Proof</h2><p>Not content with merely claiming that it is compile-time accessible, I prove that it is! Then I double down by comparing it to pure structure access. That’s right! While the dictionary sub-class with its overloaded functions can be used just like a normal <code>dict</code>, and all its elements, including the permanent, proxied-typed ones can be found by normal look-up or by iteration, it is just as fast as a <b>raw struct</b>! See for yourself:</p><pre><code> struct rock_struct { int64 rock = 42; };</code></pre><p>In the “fast-dict” branch are unit-tests, including two dummy ones that were used solely to compare the code generation of the sub-dict and the structure. I captured the disassembly of both for your viewing pleasure: as advertised, each is just as fast as the other!</p><pre><code> d1.rock = rand();call qword ptr [rand] movsxd rcx,eax mov qword ptr [rsp+38h],rcx use_rock(d1);lea rcx,[d1] call dak::use_rock std::wcout &lt;&lt; d1.rock;mov rbx,qword ptr [d1] mov rcx,qword ptr [std::wcout] mov rdx,rbx call qword ptr [operator&lt;&lt;] d1.rock += rand();call qword ptr [rand] movsxd rcx,eax add rbx,rcx use_rock(d1);lea rcx,[d1] mov qword ptr [d1],rbx call dak::use_rock std::wcout &lt;&lt; d1.rock;mov rdx,qword ptr [d1] mov rcx,qword ptr [std::wcout] call qword ptr [operator&lt;&lt;] use_rock(d1);lea rcx,[d1] call dak::use_rock</code></pre><pre><code> d1[voc::rock] = rand();call qword ptr [rand] movsxd rcx,eax mov qword ptr [rsp+38h],rcx use_rock(d1);lea rcx,[d1] call dak::use_rock std::wcout &lt;&lt; d1[voc::rock];lea rdx,[rsp+38h] mov rcx,qword ptr [std::wcout] call dak::operator&lt;&lt; d1[voc::rock] += rand();call qword ptr [rand] add eax,dword ptr [rsp+38h] movsxd rcx,eax mov qword ptr [rsp+38h],rcx use_rock(d1);lea rcx,[d1] call dak::use_rock std::wcout &lt;&lt; d1[voc::rock];lea rdx,[rsp+38h] mov rcx,qword ptr [std::wcout] call dak::operator&lt;&lt; use_rock(d1);lea rcx,[d1] call dak::use_rock</code></pre><h2>Conclusion</h2><p>We looked down the fairway, setting our sights on the dictionary access and trying to see how low par for the course could go. And lo and behold!, how low we went!</p><p>But I feel you are tense, confused and cross..</p><p>This is a parody of design, an abomination! Deriving from a <code>dict</code> class? Would you derive from a <code>std::vector</code>, a <code>std::map</code>, a <code>std::pair</code>? What kind of respectable golfer would ever do that? And I’d agree! (Wait, what? Who said that?) No, no, no, no, I would really agree! I would, I would, I would, except…</p><p>… you see, everything in life is a question of perspective. Of how we perceive the world. And in the case of programming, perception is all about naming. Naming types, naming functions, naming variables. What’s in a name? Names shape our vision of the world, and on this much smaller scale, of our designs. So… what if I told you <code>dict</code> is not the real name of the class? What would happen if we renamed it … <code>object</code>?</p><p>Ah, the final key! Yes, now it makes sense to derive from <code>object</code>. Now it makes sense to add fixed permanent elements to an <code>object</code> to hold values of fixed type! It’s no longer even a surprising design. It’s basically how a language like Python works under the hood. In Python, every object of every class is really just a dictionary of values indexed by name. It’s just that now you can do it directly in C++.</p><p>It’s also very useful. You no longer need to write and rewrite boiler-plate code for every <code>struct</code> and <code>class</code>. You can have a single implementation for all types for things like serialization, undo/redo, data lookup, hooking-up to UI elements, and many other activities I’m sure you can think up. You write it once for the <del><code>dict</code></del> <code>object</code> class. Every sub-class inherits the implementation of <code>object</code> and all data is always accessible via simple iteration over elements or name lookup.</p><p>Ain’t that something? So, did we birdie this one or not?</p></div>

<h3>I was obsessed with cardboard as a kid. I could build anything.</h3> <p>Forts, trucks, spaceships… Swords were definitely a favourite.</p> <p>Cardboard was a medium I could use to build whatever I wanted. More importantly, I could build it QUICKLY.</p> <p>With some questionably safe exacto-knifing, a door appeared in a large box that became a fort, a spaceship got flaps and hatches, or the cardboard sheet turned into He-Man’s Power Sword.</p> <div> <p>It was fast.</p> <p>It was cheap.</p> <p>It was easy.</p> </div> <p>But I think the best part was when I held the finished product in my hand.</p> <p>I could climb into the fort and shut the door. I could lift the spaceship and it would fly. I could challenge my friends with my freshly forged cardboard sword. For many reasons, the latter was my favourite.</p> <p>My friends and I would wage war on each other with our corrugated creations. Inevitably they would break — so it was into the recycling bin for the weapons and back to the drawing board for us. We’d spend hours reinforcing this piece, cutting off that part, and then back into the fray we went.</p> <p><picture><source srcset="https://mirror.spiria.com/site/assets/files/2546/cardboard-sword.400x0.webp" type="image/webp" media="(max-width: 599px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/cardboard-sword.760x0.webp" type="image/webp" media="(max-width: 999px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/cardboard-sword.1039x0.webp" type="image/webp" media="(min-width: 1000px)" /><img title="Cardboard sword." src="https://mirror.spiria.com/site/assets/files/2546/cardboard-sword.webp" alt="Cardboard sword." /></picture></p> <h3>So, I guess it isn’t shocking that I love paper prototyping.</h3> <p>I was introduced to paper prototyping in my Interactive Multimedia post-grad at Sheridan College. In one of our first UX classes, our instructor showed us how quickly we could create an app with a marker and a notepad.</p> <p>I was hooked.</p> <p>I had a preconceived notion about design being a long, drawn-out process with rough drafts, concepts, approvals, etc. This approach did away with all that. All that mattered was finding the best way for the user to accomplish a specific goal.</p> <h3>So, what is a paper prototype?</h3> <p>A paper prototype is a disposable design that is built to solve a problem. It is a series of frames roughly drawn out in such a way as to represent an application at its most basic level. When you’re creating a paper prototype, you’re building the interaction — not focusing on the design.</p> <p><picture><source srcset="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-01.400x0.webp" type="image/webp" media="(max-width: 599px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-01.760x0.webp" type="image/webp" media="(max-width: 999px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-01.1039x0.webp" type="image/webp" media="(min-width: 1000px)" /><img title="Kevin’s prototype." src="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-01.webp" alt="Kevin’s prototype." /></picture></p> <p><picture><source srcset="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-02.400x0.webp" type="image/webp" media="(max-width: 599px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-02.760x0.webp" type="image/webp" media="(max-width: 999px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-02.1039x0.webp" type="image/webp" media="(min-width: 1000px)" /><img title="Kevin’s prototype." src="https://mirror.spiria.com/site/assets/files/2546/kevin-proto-02.webp" alt="Kevin’s prototype." /></picture></p> <p>Each page represents a different screen or step in a flow. So if you were creating a prototype for a registration process, you might do something like this:</p> <div> <p>Screen 1: Welcome Screen</p> <p>Screen 2: Email and password, privacy policy</p> <p>Screen 3: Name, profile picture, age, bio, location, phone number</p> <p>Screen 4: Thank you</p> <p>Screen 5: Application home page</p> </div> <p>There, you’ve got a basic flow. Each page gets only the essential elements it needs — buttons, text, a quick shape to represent an illustration. Now it’s time to test it.</p> <p>Your first “test” is when you, yourself, are creating it. When you’re holding a paper prototype, you can feel it in your hand. Take yourself through the whole flow, interacting with the prototype as you would with a real device. Each time you tap a button, swap out the current screen for the next one that you’d expect in the flow. Repeat the process until you’re comfortable with how everything is coming together.</p> <p>Now, grab some extra materials and find some users to test your prototype.</p> <p>Show users your prototype. Ask them to interact with it. As they tap, scroll and swipe, bring in the next piece of paper. You are the operating system. At the same time, you get to see the problems the users are having.</p> <p>Between each user test, go through your flow again. Find the screens your users had trouble with. Pull them out. Break them down. Simplify. Scribble out a new screen and swap it in. Does it make sense? Do you need to adjust any other screens because of this change? Now, test it again.</p> <p>The beauty of this approach is how quickly you can add and remove elements. Paper prototypes don’t need to be uploaded or deployed. They don’t rely on internet access and they don’t crash. They help find problems in real-time with very little time and cost outlays.</p> <p><picture><source srcset="https://mirror.spiria.com/site/assets/files/2546/istock-898079432.400x0.webp" type="image/webp" media="(max-width: 599px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/istock-898079432.760x0.webp" type="image/webp" media="(max-width: 999px)" /><source srcset="https://mirror.spiria.com/site/assets/files/2546/istock-898079432.1039x0.webp" type="image/webp" media="(min-width: 1000px)" /><img title="Paper prototype." src="https://mirror.spiria.com/site/assets/files/2546/istock-898079432.webp" alt="Paper prototype." /></picture></p> <h3>So grab a marker and a notepad and get started!</h3>

<div><p>When faced with a sluggish program, first you take slowness measurements, then you hone in on the problem, and finally you look for a solution. In this article I’ll explain how I saw, measured, located and fixed the problem.</p><p>Reality check: I won’t be presenting the kind of amazing, low-level, well-researched, mathematically-proven technique of the likes of Daniel Lemire. For an example of that, see: “<a href="https://lemire.me/blog/2018/05/03/how-fast-can-you-parse-json/">How fast can you parse JSON?</a>”.</p><p>In this article, I won’t be counting CPU cycles. I’ll simply talk about the simple optimizations an average software programmer can do as part of his routine work.</p><h2>Seeing the Problem</h2><p>The program I was working on was a filter for large text files organized as a tree, where the indentation of each line in the file represented the nesting level. And by large files, I mean files routinely processing 100MB to 1GB of log data. This program was written in C++.</p><p>The first, simple approach was to read the text into a wstring vector (std::vector&lt;std::wstring&gt;). This starting point was based on the well-known principles of choosing a container in C++. The first principle is:</p><p><i><b>Always use a std::vector</b></i>.</p><p>(The second principle is “use a sorted vector” and the third is “are you really sure a vector won’t do?”)</p><p>It became obvious, when testing with such large files, that both reading and filtering such a large vector of strings was slow going. When it takes nothing more than a stopwatch to tell that the performance just isn’t there, you’re allowed to peek under the hood.</p><h2>Measuring the Problem</h2><p>Fortunately, taking that peek is very easy in Visual Studio, as even the community edition comes with a very decent CPU profiler. Right under the “Debug” menu, the entry “Performance Profiler...” (short-cut: Alt+F2) will open a window with your default project target already selected for profiling. From there, just click the “Start” button and your program will run under the CPU profiler. Simply select the operation, measure, and quit. Measurement done!</p><p>Here is what the profiler window looks like while it is recording your program activities:</p><p><picture><source type="image/webp" srcset="https://mirror.spiria.com/site/assets/files/4422/profiler_1.400x0.webp" media="(max-width: 599px)"><source type="image/webp" srcset="https://mirror.spiria.com/site/assets/files/4422/profiler_1.760x0.webp" media="(max-width: 999px)"><source type="image/webp" srcset="https://mirror.spiria.com/site/assets/files/4422/profiler_1.1039x0.webp" media="(min-width: 1000px)"><img src="https://mirror.spiria.com/site/assets/files/4422/profiler_1.webp" style="width: 100%; border-style:solid; border-width:1px;" alt="Profiler" title="Profiler"></source></source></source></picture></p><h2>Locating the Problem</h2><p>The profiler will analyze the data and present you with a list of locations where your program is spending most of its time. By default, this list is organized by total CPU time, as follows:</p><p><picture><source type="image/webp" srcset="https://mirror.spiria.com/site/assets/files/4422/profiler_2.400x0.webp" media="(max-width: 599px)"><source type="image/webp" srcset="https://mirror.spiria.com/site/assets/files/4422/profiler_2.760x0.webp" media="(max-width: 999px)"><source type="image/webp" srcset="https://mirror.spiria.com/site/assets/files/4422/profiler_2.1039x0.webp" media="(min-width: 1000px)"><img src="https://mirror.spiria.com/site/assets/files/4422/profiler_2.png" style="width: 100%; border-style:solid; border-width:1px;" alt="Profiler" title="Profiler"></source></source></source></picture></p><p>While this gives you a bird’s eye view, it also means you see a lot of irrelevant data. You can easily find the problem area by looking for big jumps in the percentage of time spent. In this case, it turns out that a lot of time is spent copying text strings and allocating memory. The memory allocations come from the text strings and from resizing the vector they are kept in. There is also a significant amount of time spent in file I/O when reading the text. Now that we have a few clues, we’re in a good position to figure out how to solve the performance problem.</p><h2>Borrowing a Few Tricks</h2><p>A good starting point when optimizing is knowing what tricks others have used. One obvious thing to remember is that filtering doesn’t change the text; it only changes what is kept or not. That means we can borrow a few tips from functional languages:</p><ul> <li><b><i>Constants can be shared</i></b>.</li> <li><b><i>Avoid moving data</i></b>.</li> <li><b><i>Delegate managing memory</i></b>.</li></ul><p>The trick is to make the text shareable, to take it from disk as directly as possible and to take away the management of the memory it resides in. So, we will do the following:</p><ol> <li><b><i>Constants can be shared</i></b>: read the data from disk in large buffers of non-modifiable data.</li> <li><b><i>Avoid moving data</i></b>: scan these buffers for text lines and keep pointers directly in the buffer.</li> <li><b><i>Avoid moving data</i></b>: share these buffers and text lines in the result of filtering.</li> <li><b><i>Delegate managing memory</i></b>: share these buffers when making copies of the text.</li></ol><p>In short, we’ve gone from the code:</p><pre><code>struct VectorOfWStringTextHolder { using Line = std::wstring; std::vector&lt;Line&gt; Lines; };</code></pre><p>To the code:</p><pre><code>struct VectorOfSharedBuffersTextHolder { using Buffer = vector&lt;wchar_t&gt;; using BufferPtr = shared_ptr&lt;Buffer&gt;; using Buffers = vector&lt;BufferPtr&gt;; Buffers TextBuffers; using Line = wchar_t *; std::vector&lt;Line&gt; Lines;</code></pre><p>The idea here is to use a vector of shared buffers so that the addresses of the buffers never change, and so that they can easily be shared between multiple instances.</p><p>Of course, these changes have made the code more complex. However, this complexity can be hidden from view, as the buffers can be kept private while only a vector of lines is exposed publicly. In this instance, the code was simplified to make sure the benchmarks measured what we wanted to measure.</p><h2>The Results</h2><p>Yes, you did read “benchmarks”! After all, there is no point in optimizing if you cannot verify the improvement. Usually, you would simply reuse the CPU profiler and see the new results. However, for the purposes of this article, I went further and extracted the gist of the old and new code and wrote a small benchmark program that loads a large file and makes partial copies of the text lines to mimic the filtering.</p><p>The advantage of writing an explicit benchmark is that you can put the timing measurements exactly where you want them, in order to measure only what you intend to. With CPU profiling, the whole program is measured and it is harder to extract the relevant data.</p><p>Here are the results. (I excluded the I/O times, but reading in large buffers and parsing the buffers directly gave a similar gain.)</p><table style="width:100%"> <tbody> <tr> <th>Data Format</th> <th>Time</th> </tr> <tr> <td style="vertical-align: top;"><i>Vector of strings</i></td> <td style="vertical-align: top;">13.28s</td> </tr> <tr> <td style="vertical-align: top;"><i>Shared buffers of shared lines</i></td> <td style="vertical-align: top;">1.2s</td> </tr> </tbody></table><p>The above shows a tenfold improvement in speed. But in actual fact, we got even more improvement on larger files, as re-allocating a very large vector is very expensive.</p><h2>The Code</h2><p>Below, I have provided the code showing various benchmarks, as well as another approach using std::deque. I also did some benchmarks in a tree structure that I haven’t mentioned in this article, since the change did not improve performance, but avoided another unrelated problem having to do with recursion depth and stack overflow when destroying deeply linked structures of shared pointers.</p><p>You can find the code <a href="https://github.com/pierrebai/FastTextContainer">on GitHub</a>.</p><p>Note that the real use case for this was in my text tree filtering project, also available on GitHub. The optimization is done in the TextTree and holds the text data through a TextHolder. It also uses a similar trick to have stable addresses for its tree nodes: the (often-overlooked) fact is that a std::deque does not invalidate pointers when appended to. The code is <a href="https://github.com/pierrebai/TreeReader">here</a>.</p></div>

<div><p>At the same time, I wanted to be able to drag these items between multiple list widgets. This is easily supported in the standard list widget. Drag-and-drop allows you to design a user interface that is easy to use and intuitive.</p><p>It turns out that fulfilling these two requirements proved to be more difficult than I anticipated. In this article, I’ll outline all the problems I encountered, and tell you the final, surprisingly simple answer to all those problems.</p><h2>First Attempt</h2><p>The first, naïve, approach is to assume that it is possible to simply add a QWidget containing multiple sub-widgets in a QListWidget. After all, there are many examples of list views containing check-boxes. Therefore, adding other types of buttons must simply require finding the right function to call. Something about it must be buried somewhere in the Qt documentation, I thought.</p><p>I carefully sifted through the documentation, but always came up empty. Turns out, this is just plain impossible.</p><p>When you think about it, there is a very sensible reason why Qt does not support this. A Qt list view can contain thousands of items in a single view. If each of those items contained multiple live widgets, it could add up to thousands of live widgets. Given that each widget provides a rich interface, and thus a lot of data, you’d end up with too heavy a load. Instead, Qt only keeps the barest minimum amount of data per item. This design choice means that widgets cannot be put directly in a list view.</p><h2>Second Attempt</h2><p>Digging in the Qt documentation, I discovered the concept of the item delegate. This is how Qt allows the editing of data in a list view item. Unfortunately, this solution is far from ideal, because the buttons are not visible in the list view. To make the buttons visible, the user must invoke the item editor through what Qt calls an edit action: either pressing F2 or double-clicking. But these are not actions a user would know. The resulting user interface is neither intuitive nor easy to discover, making it inelegant.</p><p>Besides, this requires the user to know the item is editable in the first place.</p><p>If only there was a way to show the editable form at all times…</p><h2>Third Attempt</h2><p>… but there is! The item delegate provides a paint function that can be modified to paint anything. But how does one paint an arbitrary widget? There is actually a way. If you look at the source code for how the standard list widget paints the check-box, you can see that Qt provides “style-option” classes to draw widgets. Unfortunately, you’ll quickly notice that this feature is seriously under-documented and, worse, that you’ll need a lot of code to support all the normal functionalities of the widget, like hover and selection.</p><p>Then, even if you do paint the widgets, they’re still not interactive! Users still have to double-click to make them editable, which makes things even more confusing for them.</p><p>And there is yet another catch! Item delegates can only be used by custom item models, i.e. you cannot use a delegate with a standard item model. Think this is not so bad? Unfortunately, implementing your own custom model is anything but easy.</p><p>The problem is that Qt has strict rules on how an item model should behave. Getting all the rules straight is difficult, and the slightest deviation leads to strange behavior and glitches that are very hard to diagnose. And this is just for the non-editable models! If you want to support drag-and-drop, then the rules are even more complex and unforgiving.</p><p>So, choosing the route of using a delegate results in writing a lot of code that is highly complicated to get right and maintain. There must be a better way!</p><h2>Final Attempt</h2><p>So what is the correct way to put complex widgets in a list? The answer is disappointingly simple: don’t use a list widget. Use a simple widget. Yes, a straight QWidget.</p><p>After all, a QWidget can contain an arbitrary list of sub-widgets. Of course, you then have to add the standard functionality of a list view: things like selection, drag-and-drop and scrolling. But it turns out that this is the simplest of all the approaches. And, as a bonus, you are assured that the behavior of all user interfaces you put in your items will be exactly the same as the normal behavior the user is expecting.</p><h2>Solution</h2><p>My solution to the problem of making a Qt list widget that can contain complex widgets is as follows. Four classes interact with each other to provide the functionality:</p><table style="width:100%"> <tbody> <tr> <th>Class</th> <th>Purpose</th> </tr> <tr> <td style="vertical-align: top;"><i>QWidgetListWidget</i></td> <td style="vertical-align: top;">The widget that contains the items.</td> </tr> <tr> <td style="vertical-align: top;"><i>QWidgetListItem</i></td> <td style="vertical-align: top;">The items that can be put in the list. Can contain any number of sub-widgets, of any type.</td> </tr> <tr> <td style="vertical-align: top;"><i>QWidgetScrollListWidget</i></td> <td style="vertical-align: top;">A wrapper around QWidgetListWidget to provide scrolling.</td> </tr> <tr> <td style="vertical-align: top;"><i>QWidgetListMimeData</i></td> <td style="vertical-align: top;">The MIME data used to support drag-and-drop between multiple QWidgetListWidget.</td> </tr> </tbody></table><h3>QWidgetListWidget</h3><p>This is one of the two main points of interest in the design: the list widget. This list view supports item selection and drag-and-drop. It provides a simple interface, made of a few functions. Here is the C++ declaration of these functions.</p><pre><code>// Create a widget list widget.QWidgetListWidget(ListModifiedCallbackFunction modifCallback, bool stretch, QBoxLayout::Direction dir, QWidget * parent);// Check if the list is vertical or horizontal.bool isVertical() const;// Clears the list panel of all items.void clear();// Add a widget item.QWidgetListItem* addItem(QWidgetListItem* item, int index = -1);// Remove a widget item.void removeItem(QWidgetListItem* item);// Retrieve all widget items kept directly in this list widget.std::vector&lt;QWidgetListItem*&gt; getItems(bool onlySelected = false) const;// Retrieve all selected widget items kept directly in this list widget.std::vector&lt;QWidgetListItem*&gt; getSelectedItems() const;</code></pre><h3>QWidgetLisItem</h3><p>This is the second main point of interest. It is a simple item that can be selected and cloned. The cloning is used during drag-and-drop to copy an item from one list to another. Here is the full C++ interface, which is quite short.</p><pre style="margin-bottom: 20em;"><code>// Create an item.QWidgetListItem(QWidget* parent);// Selection.bool isSelected() const;void select(bool sel);// Item cloning for drag-and-drop.virtual QWidgetListItem* clone() const;</code></pre><h3>QWidgetScrollListWidget</h3><p>This class only exists to make scrolling in the list widget optional. It might seem strange to make scrolling optional, but it is quite handy when you want to embed a list view within items in another list view. The whole C++ interface is simply this:</p><pre style="margin-bottom: 20em;"><code>// Create a scrollable widget around another widget.QWidgetScrollListWidget(QWidget * widget, QWidget* parent);</code></pre><h3>QWidgetListMimeData</h3><p>The final class exists to support drag-and-drop. You should never have to use it directly. The entirety of its implementation in C++ is as follows:</p><pre style="margin-bottom: 20em;"><code>static constexpr char MimeType[] = "application/x-qwidget-list-item";QWidgetListItem* Widget = nullptr;QPoint HotSpot;</code></pre><h2>The Code</h2><p>The entirety of the implementation is <a href="https://github.com/pierrebai/QWidgetListWidget">available on GitHub</a>.</p><p>The code comes with an example application showing how to use the classes. See the description of how to build the project on GitHub.</p><p>More complex example can be found in the TreeFilterApp project. It even shows list widgets within another list widget. The code is <a href="https://github.com/pierrebai/TreeReader">also on GitHub</a>.</p></div>

<div><p>Sooner or later, someone will read your code and try to understand its purpose. That someone could even be you! And you might be puzzled by what you wrote just a few months earlier, either because it’s no longer fresh in your mind, or because you didn’t make it easy to read.</p><p>I had the opportunity to attend a presentation by Jason McCreary (@<a href="https://twitter.com/gonedark">gonedark</a>) on best practices for writing code. While none of it was revolutionary, I found it was a good reminder of code-writing ground rules.</p><p>Jason made a list of 10 coding practices to keep in mind so that you, or a future programmer, can swiftly and intuitively follow your code. Look at it this way: if your code is immediately understandable, down the line, it could save the week-end of a post-production support colleague – or even your own.</p><h2>Formatting</h2><p>So many standards, so many possibilities. Who cares about tabs versus spaces, or Allman versus K&amp;R. The main thing is to adopt a standard (for example, <a href="https://www.php-fig.org/psr/psr-12/">PSR12</a>), then stick to it. Code sniffers and IDEs are great at detecting code standard violations and help to maintain a clean, consistent codebase. Be exacting, be consistent.</p><h2>Dead code</h2><p>If your code is dead, delete it!!! Go through all those commented blocks, unused variables and unreachable code. Treat your program like a tree: cut off the deadwood to support new growth and avoid risk.</p><h2>Nested code</h2><p>When following principle number one, if your code ends up so deeply nested that it starts beyond your screen width, you should review your method. Of course, nested conditions are part and parcel of any code, but, even if you write sub-functions, remember that future programmers may not have access to the diagram you had in front of you to code your conditions! Unravel nested code by using guard clauses, early returns or aspects of functional programming.</p><h2>Using objects</h2><p>This primarily concerns PHP programmers, but also applies to other languages. The idea is to use objects instead of tables. The most obvious example in PHP is using tables to pass a variable number of variables.</p><h2>Big blocks</h2><p>We all know that a 500-line block is just wrong. So don’t do it! Ever! If your block reaches a critical length, take the opportunity to refactor it into a more readable, less complex block.</p><h2>Naming</h2><p>“Foo” is a variable name for classroom exercises, not for real-life programming.</p><p>If your <code>getWeekReport</code> function provides a monthly report, it’s misnamed.</p><p>If your <code>getReport</code> function provides a weekly report, then you should rename it <code>getWeekReport</code>.</p><p>If you can’t think of a name, just continue coding, and leave it until later. Never run after names: they’ll come to you in time.</p><h2>Removing comments</h2><p>This may be rocking the boat, but consider the following:</p><ol> <li>DocBlocks are not comments; we want them and need them.</li> <li>Code that needs to be explained, needs to be improved.</li> <li><code>$today = date(); // Set today date</code>: a classic example of unnecessary comments.</li> <li>Comments are sometimes justified; for example, when they explain a business rule.</li> <li>Better no comment than a faulty comment!</li> <li>At the end of the day, comments don’t matter, since only code is executed. So only code really matters.</li></ol><p>Challenge yourself to re-write your code so it needs no comments.</p><h2>Reasonable returns</h2><p>Code readers must be able to easily understand what the function returns. Some prefer one single return per function. Others group functions by type of return (for example, all the <code>if … return true</code> functions at the beginning, then all the <code>if return false</code> functions).</p><p>Don’t kick problems down the line; for example, <code>return null</code> functions, which mean that the <code>null</code> case will have to be dealt with later on.</p><h2>Rule of three</h2><p>This was a new concept for me, and I found it interesting. This is how it goes:</p><ul> <li>If I ask you what comes after “2”, you’ll be hard-pressed to find the answer. It could be 3, 4, or even 1 or 2.1.</li> <li>If I ask you what comes after “2, 4”, you might hazard a guess.</li> <li>But if I ask you what comes after “2, 4, 16”, you’ll figure out it’s 256.</li></ul><p>That’s the rule of three.</p><p>In programming terms, it means that your code should follow a set logic. Avoid code that goes “2, 4, 6, 27”. It might be right and do what it needs to do, but it’s counterintuitive for a human reader.</p><p>n the same vein, if your code goes “2, 4, <code>call function DonextStep</code>”, it could be unreadable. It means that you may have to duplicate your code. This runs counter to the <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a> principle... ¯\_(ツ)_/¯</p><h2>Symmetry</h2><p>To keep it simple, let’s just say that if you have two functions, <i>create</i> and <i>update</i>, then they should always follow the same parameters and return the same type of variable.</p><pre><code>Function update (User user) : boolean successFunction create(User user) : boolean success</code></pre><p>Rather than:</p><pre><code>Function update (int userId, User user) : boolean successFunction create(Array[]) : User user</code></pre><p>If you’d like to dig deeper, Jason McCreary has released BaseCode, a guide in which he elaborates on these principles and illustrates them with many practical examples (available for $29 on <a href="https://basecodefieldguide.com">his site</a>). You can also take a look at the screencasts on his <a href="https://www.youtube.com/watch?v=s9LwS6RFax0&amp;list=PLmwAMIdrAmK7cjLLYrKppUhaR2ywTuKhm&amp;index=1">YouTube channel</a>.</p><p>Happy coding!</p></div>