TechOverflow -

Apr

2024

How to fix pandas / matplotlib error: raise ValueError(f’Date ordinal {x} converts to {dt}..

Problem:

You are trying to plot a pandas DataFrame or Series using code such as

df.plot()

but you see an error message like

Traceback (most recent call last):
  ...
  File "/usr/local/lib/python3.11/site-packages/matplotlib/figure.py", line 3390, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/backend_bases.py", line 2164, in print_figure
    self.figure.draw(renderer)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/artist.py", line 95, in draw_wrapper
    result = draw(artist, renderer, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/matplotlib/figure.py", line 3154, in draw
    mimage._draw_list_compositing_images(
  File "/usr/local/lib/python3.11/site-packages/matplotlib/image.py", line 132, in _draw_list_compositing_images
    a.draw(renderer)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/matplotlib/axes/_base.py", line 3034, in draw
    self._update_title_position(renderer)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/axes/_base.py", line 2969, in _update_title_position
    bb = ax.xaxis.get_tightbbox(renderer)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/axis.py", line 1334, in get_tightbbox
    ticks_to_draw = self._update_ticks()
  File "/usr/local/lib/python3.11/site-packages/matplotlib/axis.py", line 1276, in _update_ticks
    major_labels = self.major.formatter.format_ticks(major_locs)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/ticker.py", line 216, in format_ticks
    return [self(value, i) for i, value in enumerate(values)]
  File "/usr/local/lib/python3.11/site-packages/matplotlib/ticker.py", line 216, in <listcomp>
    return [self(value, i) for i, value in enumerate(values)]
  File "/usr/local/lib/python3.11/site-packages/matplotlib/dates.py", line 649, in __call__
    result = num2date(x, self.tz).strftime(self.fmt)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/dates.py", line 543, in num2date
    return _from_ordinalf_np_vectorized(x, tz).tolist()
  File "/usr/local/lib/python3.11/site-packages/numpy/lib/function_base.py", line 2372, in __call__
    return self._call_as_normal(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/numpy/lib/function_base.py", line 2365, in _call_as_normal
    return self._vectorize_call(func=func, args=vargs)
  File "/usr/local/lib/python3.11/site-packages/numpy/lib/function_base.py", line 2455, in _vectorize_call
    outputs = ufunc(*inputs)
  File "/usr/local/lib/python3.11/site-packages/matplotlib/dates.py", line 362, in _from_ordinalf
    raise ValueError(f'Date ordinal {x} converts to {dt} (using

Solution:

This issue has been discussed on the Matplotlib Github and also pandas Github. This issue occurs if you use df.plot() but then set custom formatting options for the x axis, for datetime index DataFrames.

Right now you can work around this bug by manually plotting the data:

for column in df.columns:
    plt.plot(df.index.values, df[column].values, label=column)

instead, but keep in mind that you might need to set some matplotlib options yourself (which pandas would otherwise set automatically) when plotting everything manually.

Apr

2024

How to fix ESP32 ESP_LOGI(…) error: expected ‘)’ before ‘msg’

Problem:

You have code in your project such as

ESP_LOGI("MyTag", msg);

where msg is a const char*, but the project fails to compile with an error message such as

src/main.cpp:280:26: error: expected ')' before 'msg'
  280 |         ESP_LOGI("MyTag", msg);

Solution:

The format parameter can’t be any string but needs to be a string constant. Replace your code by

ESP_LOGI("MyTag", "%s", msg);

Apr

2024

gitlab-rake: How to list tasks

gitlab-rake --tasks

or, on Docker/docker-compose:

docker-compose exec gitlab gitlab-rake --tasks

Example output

(for Gitlab 14)

rake about                                                                                          # List versions of all Rails frameworks and the environment
rake acts_as_taggable_on:sharded_db:install_initializer                                             # Install initializer setting custom base class
rake acts_as_taggable_on_engine:install:migrations                                                  # Copy migrations from acts_as_taggable_on_engine to application
rake acts_as_taggable_on_engine:tag_names:collate_bin                                               # Forcing collate of tag names to utf8_bin
rake acts_as_taggable_on_engine:tag_names:collate_ci                                                # Forcing collate of tag names to utf8_general_ci
rake app:template                                                                                   # Applies the template supplied by LOCATION=(/path/to/template) or URL
rake app:update                                                                                     # Update configs and some other initially generated files (or use just update:configs or update:bin)
rake assets:clean[keep]                                                                             # Remove old compiled assets
rake assets:clobber                                                                                 # Remove compiled assets
rake assets:environment                                                                             # Load asset compile environment
rake assets:precompile                                                                              # Compile all the assets named in config.assets.precompile
rake cache:clear:redis                                                                              # GitLab | Cache | Clear redis cache
rake cache_digests:dependencies                                                                     # Lookup first-level dependencies for TEMPLATE (like messages/show or comments/_comment.html)
rake cache_digests:nested_dependencies                                                              # Lookup nested dependencies for TEMPLATE (like messages/show or comments/_comment.html)
rake ci:cleanup:builds                                                                              # GitLab | CI | Clean running builds
rake config_lint                                                                                    # Checks syntax for shell scripts and nginx config files in 'lib/support/'
rake css:build                                                                                      # Build your CSS bundle
rake css:clobber                                                                                    # Remove CSS builds
rake css:install                                                                                    # Install JavaScript dependencies
rake css:install:bootstrap                                                                          # Install Bootstrap
rake css:install:bulma                                                                              # Install Bulma
rake css:install:postcss                                                                            # Install PostCSS
rake css:install:sass                                                                               # Install Sass
rake css:install:tailwind                                                                           # Install Tailwind
rake danger_local                                                                                   # Run local Danger rules
rake db:create                                                                                      # Creates the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:create:...
rake db:drop                                                                                        # Drops the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:drop:all ...
rake db:encryption:init                                                                             # Generate a set of keys for configuring Active Record encryption in a given environment
rake db:environment:set                                                                             # Set the environment value for the database
rake db:fixtures:load                                                                               # Loads fixtures into the current environment's database
rake db:migrate                                                                                     # Migrate the database (options: VERSION=x, VERBOSE=false, SCOPE=blog)
rake db:migrate:down                                                                                # Runs the "down" for a given migration VERSION
rake db:migrate:redo                                                                                # Rolls back the database one migration and re-migrates up (options: STEP=x, VERSION=x)
rake db:migrate:status                                                                              # Display status of migrations
rake db:migrate:up                                                                                  # Runs the "up" for a given migration VERSION
rake db:obsolete_ignored_columns                                                                    # Show a list of obsolete `ignored_columns`
rake db:prepare                                                                                     # Runs setup if database does not exist, or runs migrations if it does
rake db:reset                                                                                       # Drops and recreates all databases from their schema for the current environment and loads the seeds
rake db:rollback                                                                                    # Rolls the schema back to the previous version (specify steps w/ STEP=n)
rake db:schema:cache:clear                                                                          # Clears a db/schema_cache.yml file
rake db:schema:cache:dump                                                                           # Creates a db/schema_cache.yml file
rake db:schema:dump                                                                                 # Creates a database schema file (either db/schema.rb or db/structure.sql, depending on `ENV['SCHEMA_FORM...
rake db:schema:load                                                                                 # Loads a database schema file (either db/schema.rb or db/structure.sql, depending on `ENV['SCHEMA_FORMAT...
rake db:seed                                                                                        # Loads the seed data from db/seeds.rb
rake db:seed:replant                                                                                # Truncates tables of each database for current environment and loads the seeds
rake db:seed_fu                                                                                     # Loads seed data for the current environment
rake db:setup                                                                                       # Creates all databases, loads all schemas, and initializes with the seed data (use db:reset to also drop...
rake db:version                                                                                     # Retrieves the current schema version number
rake dev:fixtures:load[fixture_name]                                                                # GitLab | Dev | Load specific fixture
rake dev:load                                                                                       # GitLab | Dev | Eager load application
rake dev:setup                                                                                      # GitLab | Dev | Setup developer environment (db, fixtures)
rake dev:terminate_all_connections                                                                  # Used to drop all connections in development
rake file_hooks:validate                                                                            # Validate existing file hooks
rake gems:error_tracking_open_api:generate                                                          # Generate OpenAPI client for Error Tracking
rake gettext:compile                                                                                # Compile po files to json, for usage in the frontend
rake gettext:lint                                                                                   # Lint all po files in `locale/
rake gettext:regenerate                                                                             # Regenerate gitlab.pot file
rake gettext:updated_check                                                                          # Check whether gitlab.pot needs updates, used during CI
rake gitlab:app:check                                                                               # GitLab | App | Check the configuration of the GitLab Rails app
rake gitlab:artifacts:check                                                                         # GitLab | Artifacts | Check integrity of uploaded job artifacts
rake gitlab:artifacts:migrate                                                                       # GitLab | Artifacts | Migrate files for artifacts to comply with new storage format
rake gitlab:assets:check_page_bundle_mixins_css_for_sideeffects                                     # GitLab | Assets | Check that scss mixins do not introduce any sideffects
rake gitlab:assets:clean                                                                            # GitLab | Assets | Clean up old compiled frontend assets
rake gitlab:assets:compile                                                                          # GitLab | Assets | Compile all frontend assets
rake gitlab:assets:fix_urls                                                                         # GitLab | Assets | Fix all absolute url references in CSS
rake gitlab:assets:hash_sum                                                                         # GitLab | Assets | Return the hash sum of all frontend assets
rake gitlab:assets:purge                                                                            # GitLab | Assets | Remove all compiled frontend assets
rake gitlab:assets:purge_modules                                                                    # GitLab | Assets | Uninstall frontend dependencies
rake gitlab:assets:vendor                                                                           # GitLab | Assets | Compile vendor assets
rake gitlab:background_migrations:finalize[job_class_name,table_name,column_name,job_arguments]     # Synchronously finish executing a batched background migration
rake gitlab:background_migrations:status                                                            # Display the status of batched background migrations
rake gitlab:backup:create                                                                           # GitLab | Backup | Create a backup of the GitLab system
rake gitlab:backup:restore                                                                          # GitLab | Backup | Restore a previously created backup
rake gitlab:banzai:render                                                                           # GitLab | Banzai | Render markdown using our FullPipeline (input will be requested)
rake gitlab:check                                                                                   # GitLab | Check the configuration of GitLab and its environment
rake gitlab:ci_secure_files:check                                                                   # GitLab | CI Secure Files | Check integrity of uploaded Secure Files
rake gitlab:ci_secure_files:migrate                                                                 # GitLab | CI Secure Files | Migrate Secure Files to remote storage
rake gitlab:cleanup:block_removed_ldap_users                                                        # GitLab | Cleanup | Block users that have been removed in LDAP
rake gitlab:cleanup:delete_orphan_job_artifact_final_objects                                        # GitLab | Cleanup | Delete orphan job artifact objects stored in the @final directory based on the CSV file
rake gitlab:cleanup:list_orphan_job_artifact_final_objects[provider]                                # GitLab | Cleanup | Generate a CSV file of orphan job artifact objects stored in the @final directory
rake gitlab:cleanup:orphan_job_artifact_files                                                       # GitLab | Cleanup | Clean orphan job artifact files in local storage
rake gitlab:cleanup:orphan_lfs_file_references                                                      # GitLab | Cleanup | Clean orphan LFS file references
rake gitlab:cleanup:orphan_lfs_files                                                                # GitLab | Cleanup | Clean orphan LFS files
rake gitlab:cleanup:project_uploads                                                                 # GitLab | Cleanup | Clean orphaned project uploads
rake gitlab:cleanup:remote_upload_files                                                             # GitLab | Cleanup | Clean orphan remote upload files that do not exist in the db
rake gitlab:cleanup:remove_missed_source_branches                                                   # GitLab | Cleanup | Clean missed source branches to be deleted
rake gitlab:cleanup:sessions:active_sessions_lookup_keys                                            # GitLab | Cleanup | Sessions | Clean ActiveSession lookup keys
rake gitlab:clickhouse:migrate[skip_unless_configured]                                              # GitLab | ClickHouse | Migrate the databases (options: VERSION=x, VERBOSE=false, SCOPE=y)
rake gitlab:clickhouse:migrate:main[skip_unless_configured]                                         # GitLab | ClickHouse | Migrate the main database (options: VERSION=x, VERBOSE=false, SCOPE=y)
rake gitlab:clickhouse:rollback:main                                                                # GitLab | ClickHouse | Rolls the main database back to the previous version (specify steps w/ STEP=n)
rake gitlab:container_registry:configure                                                            # GitLab | Container Registry | Configure
rake gitlab:db:active                                                                               # Check if there have been user additions to the database
rake gitlab:db:cells:bump_cell_sequences[increase_by]                                               # Bump sequences for cell-local tables on the cells database
rake gitlab:db:clean_structure_sql                                                                  # This adjusts and cleans db/structure.sql - it runs after db:schema:dump
rake gitlab:db:configure                                                                            # GitLab | DB | Configures the database by running migrate, or by loading the schema and seeding if needed
rake gitlab:db:create_dynamic_partitions                                                            # Create missing dynamic database partitions
rake gitlab:db:decomposition:connection_status                                                      # Check if PostgreSQL max_connections needs to be increased
rake gitlab:db:decomposition:migrate                                                                # Migrate single database to two database setup
rake gitlab:db:decomposition:rollback:bump_ci_sequences[increase_by]                                # Bump all the CI tables sequences on the Main Database
rake gitlab:db:dictionary:generate                                                                  # Generate database docs yaml
rake gitlab:db:drop_tables                                                                          # GitLab | DB | Drop all tables on all configured databases
rake gitlab:db:enqueue_reindexing_action[index_name,database]                                       # Enqueue an index for reindexing
rake gitlab:db:execute_batched_migrations                                                           # Run all pending batched migrations
rake gitlab:db:lock_writes                                                                          # GitLab | DB | Install prevent write triggers on all databases
rake gitlab:db:mark_migration_complete[version]                                                     # GitLab | DB | Manually insert schema migration version on all configured databases
rake gitlab:db:migration_testing:down:ci                                                            # Run down migrations on ci in current branch with instrumentation
rake gitlab:db:migration_testing:down:main                                                          # Run down migrations on main in current branch with instrumentation
rake gitlab:db:migration_testing:down:main_clusterwide                                              # Run down migrations on main_clusterwide in current branch with instrumentation
rake gitlab:db:migration_testing:sample_background_migrations[duration_s]                           # Sample traditional background migrations with instrumentation
rake gitlab:db:migration_testing:sample_batched_background_migrations[database,duration_s]          # Sample batched background migrations with instrumentation (legacy)
rake gitlab:db:migration_testing:sample_batched_background_migrations:ci[duration_s]                # Sample batched background migrations on ci with instrumentation
rake gitlab:db:migration_testing:sample_batched_background_migrations:main[duration_s]              # Sample batched background migrations on main with instrumentation
rake gitlab:db:migration_testing:sample_batched_background_migrations:main_clusterwide[duration_s]  # Sample batched background migrations on main_clusterwide with instrumentation
rake gitlab:db:migration_testing:up:ci                                                              # Run migrations on ci with instrumentation
rake gitlab:db:migration_testing:up:main                                                            # Run migrations on main with instrumentation
rake gitlab:db:migration_testing:up:main_clusterwide                                                # Run migrations on main_clusterwide with instrumentation
rake gitlab:db:reindex                                                                              # Reindex database without downtime to eliminate bloat
rake gitlab:db:reset_as_non_superuser[username]                                                     # Run migration as gitlab non-superuser
rake gitlab:db:schema_checker:run                                                                   # Checks schema inconsistencies
rake gitlab:db:squash[version]                                                                      # GitLab | DB | squash | squash as of a version
rake gitlab:db:truncate_legacy_tables:ci[min_batch_size]                                            # GitLab | DB | Truncate Main Tables on CI
rake gitlab:db:truncate_legacy_tables:main[min_batch_size]                                          # GitLab | DB | Truncate CI Tables on Main
rake gitlab:db:unattended                                                                           # GitLab | DB | Run database migrations and print `unattended_migrations_completed` if action taken
rake gitlab:db:unlock_writes                                                                        # GitLab | DB | Remove all triggers that prevents writes from all databases
rake gitlab:db:validate_config                                                                      # Validates `config/database.yml` to ensure a correct behavior is configured
rake gitlab:dependency_proxy:migrate                                                                # GitLab | Dependency Proxy | Migrate dependency proxy files to remote storage
rake gitlab:docs:check_deprecations                                                                 # Check that the deprecation documentation is up to date
rake gitlab:docs:compile_deprecations                                                               # Generate deprecation list from individual files
rake gitlab:docs:redirect[old_path,new_path]                                                        # GitLab | Docs | Create a doc redirect
rake gitlab:doctor:reset_encrypted_tokens                                                           # GitLab | Reset encrypted tokens for specific models
rake gitlab:doctor:secrets                                                                          # GitLab | Check if the database encrypted values can be decrypted using current secrets
rake gitlab:env:info                                                                                # GitLab | Env | Show information about GitLab and its environment
rake gitlab:exclusive_lease:clear[scope]                                                            # GitLab | Exclusive Lease | Clear existing exclusive leases for specified scope (default: *)
rake gitlab:external_diffs:force_object_storage                                                     # Override external diffs in file storage to be in object storage instead
rake gitlab:feature_categories:index                                                                # GitLab | Feature categories | Build index page for groups
rake gitlab:git:checksum_projects                                                                   # GitLab | Git | Generate checksum of project repository refs
rake gitlab:git:fsck                                                                                # GitLab | Git | Check all repos integrity
rake gitlab:gitaly:check                                                                            # GitLab | Gitaly | Check the health of Gitaly
rake gitlab:gitaly:clone[dir,storage_path,repo]                                                     # GitLab | Gitaly | Clone and checkout gitaly
rake gitlab:gitaly:install[dir,storage_path,repo]                                                   # GitLab | Gitaly | Install or upgrade gitaly
rake gitlab:gitlab_shell:check                                                                      # GitLab | GitLab Shell | Check the configuration of GitLab Shell
rake gitlab:import:all_users_to_all_groups                                                          # GitLab | Import | Add all users to all groups (admin users are added as owners)
rake gitlab:import:all_users_to_all_projects                                                        # GitLab | Import | Add all users to all projects (admin users are added as maintainers)
rake gitlab:import:user_to_groups[email]                                                            # GitLab | Import | Add a specific user to all groups (as a developer)
rake gitlab:import:user_to_projects[email]                                                          # GitLab | Import | Add a specific user to all projects (as a developer)
rake gitlab:import_export:bump_version                                                              # GitLab | Import/Export | Bumps the Import/Export version in fixtures and project templates
rake gitlab:import_export:data                                                                      # GitLab | Import/Export | Display exported DB structure
rake gitlab:import_export:export[username,namespace_path,project_path,archive_path]                 # GitLab | Import/Export | EXPERIMENTAL | Export large project archives
rake gitlab:import_export:import[username,namespace_path,project_path,archive_path]                 # GitLab | Import/Export | EXPERIMENTAL | Import large project archives
rake gitlab:import_export:version                                                                   # GitLab | Import/Export | Show Import/Export version
rake gitlab:incoming_email:check                                                                    # GitLab | Incoming Email | Check the configuration of Reply by email
rake gitlab:incoming_email:secret:edit                                                              # GitLab | Incoming Email | Secret | Edit Incoming Email secrets
rake gitlab:incoming_email:secret:show                                                              # GitLab | Incoming Email | Secret | Show Incoming Email secrets
rake gitlab:incoming_email:secret:write                                                             # GitLab | Incoming Email | Secret | Write Incoming Email secrets
rake gitlab:ldap:rename_provider[old_provider,new_provider]                                         # GitLab | LDAP | Rename provider
rake gitlab:ldap:secret:edit                                                                        # GitLab | LDAP | Secret | Edit LDAP secrets
rake gitlab:ldap:secret:show                                                                        # GitLab | LDAP | Secret | Show LDAP secrets
rake gitlab:ldap:secret:write                                                                       # GitLab | LDAP | Secret | Write LDAP secrets
rake gitlab:lfs:check                                                                               # GitLab | LFS | Check integrity of uploaded LFS objects
rake gitlab:lfs:migrate                                                                             # GitLab | LFS | Migrate LFS objects to remote storage
rake gitlab:packages:migrate                                                                        # GitLab | Packages | Migrate packages files to remote storage
rake gitlab:password:reset[username]                                                                # GitLab | Password | Reset a user's password
rake gitlab:praefect:replicas[project_id]                                                           # GitLab | Praefect | Check replicas
rake gitlab:redis:secret:edit[instance_name]                                                        # GitLab | Redis | Secret | Edit Redis secret
rake gitlab:redis:secret:show[instance_name]                                                        # GitLab | Redis | Secret | Show Redis secret
rake gitlab:redis:secret:write[instance_name]                                                       # GitLab | Redis | Secret | Write Redis secret
rake gitlab:refresh_project_statistics_build_artifacts_size[csv_path]                               # GitLab | Refresh build artifacts size project statistics for given list of Project IDs from CSV
rake gitlab:security:update_banned_ssh_keys[git_url,output_file]                                    # GitLab | Security | Update banned_ssh_keys config file from a remote Git repository
rake gitlab:seed:ci_catalog_resources[group_path,seed_count,publish]                                # Seed CI Catalog resources
rake gitlab:seed:ci_variables_group[name,seed_count,environment_scope,prefix]                       # Seed group with CI Variables
rake gitlab:seed:ci_variables_instance[seed_count,prefix]                                           # Seed instance with CI Variables
rake gitlab:seed:ci_variables_project[project_path,seed_count,environment_scope,prefix]             # Seed project with CI Variables
rake gitlab:seed:dependencies                                                                       # GitLab | Seed | Seed a new group with dependencies
rake gitlab:seed:group_seed[subgroups_depth,username]                                               # Seed groups with sub-groups/projects/epics/milestones for Group Import testing
rake gitlab:seed:issues[project_full_path,backfill_weeks,average_issues_per_week]                   # GitLab | Seed | Seeds issues
rake gitlab:seed:project_environments[project_path,seed_count,prefix]                               # Seed project with environments
rake gitlab:seed:runner_fleet[username,registration_prefix,runner_count,job_count]                  # Seed groups with sub-groups/projects/runners/jobs for Runner Fleet testing
rake gitlab:seed:vulnerabilities[project_full_path]                                                 # GitLab | Seed | Seed a project with vulnerabilities
rake gitlab:service_desk_email:secret:edit                                                          # GitLab | Service Desk Email | Secret | Edit Service Desk Email secrets
rake gitlab:service_desk_email:secret:show                                                          # GitLab | Service Desk Email | Secret | Show Service Desk Email secrets
rake gitlab:service_desk_email:secret:write                                                         # GitLab | Service Desk Email | Secret | Write Service Desk Email secrets
rake gitlab:setup                                                                                   # GitLab | Setup production application
rake gitlab:shell:install[repo]                                                                     # GitLab | Shell | Install or upgrade gitlab-shell
rake gitlab:shell:setup                                                                             # GitLab | Shell | Setup gitlab-shell
rake gitlab:sidekiq:all_queues_yml:check                                                            # GitLab | Sidekiq | Validate that all_queues.yml matches worker definitions
rake gitlab:sidekiq:all_queues_yml:generate                                                         # GitLab | Sidekiq | Generate all_queues.yml based on worker definitions
rake gitlab:sidekiq:check                                                                           # GitLab | Sidekiq | Check the configuration of Sidekiq
rake gitlab:sidekiq:migrate_jobs:queued                                                             # GitLab | Sidekiq | Migrate jobs in queues outside of routing rules
rake gitlab:sidekiq:migrate_jobs:retry                                                              # GitLab | Sidekiq | Migrate jobs in the retry set to new queue names
rake gitlab:sidekiq:migrate_jobs:schedule                                                           # GitLab | Sidekiq | Migrate jobs in the scheduled set to new queue names
rake gitlab:sidekiq:queues:check                                                                    # GitLab | Sidekiq | Validate all_queues.yml and sidekiq_queues.yml match worker definitions
rake gitlab:sidekiq:sidekiq_queues_yml:check                                                        # GitLab | Sidekiq | Validate that sidekiq_queues.yml matches worker definitions
rake gitlab:sidekiq:sidekiq_queues_yml:generate                                                     # GitLab | Sidekiq | Generate sidekiq_queues.yml based on worker definitions
rake gitlab:smtp:secret:edit                                                                        # GitLab | SMTP | Secret | Edit SMTP secrets
rake gitlab:smtp:secret:show                                                                        # GitLab | SMTP | Secret | Show SMTP secrets
rake gitlab:smtp:secret:write                                                                       # GitLab | SMTP | Secret | Write SMTP secrets
rake gitlab:snippets:list_non_migrated                                                              # GitLab | Show non migrated snippets
rake gitlab:snippets:migrate[ids]                                                                   # GitLab | Migrate specific snippets to git
rake gitlab:snippets:migration_status                                                               # GitLab | Show whether there are snippet background migrations running
rake gitlab:tcp_check[host,port]                                                                    # GitLab | Check TCP connectivity to a specific host and port
rake gitlab:terraform_states:migrate                                                                # GitLab | Terraform | Migrate Terraform states to remote storage
rake gitlab:two_factor:disable_for_all_users                                                        # GitLab | 2FA | Disable Two-factor authentication (2FA) for all users
rake gitlab:two_factor:rotate_key:apply                                                             # GitLab | 2FA | Rotate Key | Encrypt user OTP secrets with a new encryption key
rake gitlab:two_factor:rotate_key:rollback                                                          # GitLab | 2FA | Rotate Key | Rollback to secrets encrypted with the old encryption key
rake gitlab:update_project_templates                                                                # GitLab | Update project templates
rake gitlab:update_templates                                                                        # GitLab | Update templates
rake gitlab:uploads:check                                                                           # GitLab | Uploads | Check integrity of uploaded files
rake gitlab:uploads:migrate[uploader_class,model_class,mounted_as]                                  # GitLab | Uploads | Migrate the uploaded files of specified type to object storage
rake gitlab:uploads:migrate:all                                                                     # GitLab | Uploads | Migrate all uploaded files to object storage
rake gitlab:uploads:migrate_to_local[uploader_class,model_class,mounted_as]                         # GitLab | Uploads | Migrate the uploaded files of specified type to local storage
rake gitlab:uploads:migrate_to_local:all                                                            # GitLab | Uploads | Migrate all uploaded files to local storage
rake gitlab:uploads:sanitize:remove_exif[start_id,stop_id,dry_run,sleep_time,uploader,since]        # GitLab | Uploads | Remove EXIF from images
rake gitlab:usage_data:dump_non_sql_in_json                                                         # GitLab | UsageData | Generate non SQL data for usage ping in JSON
rake gitlab:usage_data:dump_sql_in_json                                                             # GitLab | UsageData | Generate raw SQLs for usage ping in JSON
rake gitlab:usage_data:dump_sql_in_yaml                                                             # GitLab | UsageData | Generate raw SQLs for usage ping in YAML
rake gitlab:usage_data:generate                                                                     # GitLab | UsageData | Generate usage ping in JSON
rake gitlab:usage_data:generate_and_send                                                            # GitLab | UsageData | Generate usage ping and send it to Versions Application
rake gitlab:usage_data:generate_from_yaml                                                           # GitLab | UsageDataMetrics | Generate usage ping from metrics definition YAML files in JSON
rake gitlab:usage_data:generate_sql_metrics_queries                                                 # GitLab | UsageDataMetrics | Generate raw SQL metrics queries for RSpec
rake gitlab:user_management:disable_project_and_group_creation[group_id]                            # GitLab | User management | Update all users of a group with personal project limit to 0 and can_create_...
rake gitlab:web_hook:add                                                                            # GitLab | Webhook | Adds a webhook to the projects
rake gitlab:web_hook:list                                                                           # GitLab | Webhook | List webhooks
rake gitlab:web_hook:rm                                                                             # GitLab | Webhook | Remove a webhook from a namespace
rake gitlab:workhorse:install[dir,repo]                                                             # GitLab | Workhorse | Install or upgrade gitlab-workhorse
rake gitlab:x509:update_signatures                                                                  # GitLab | X509 | Update signatures when certificate store has changed
rake grape:path_helpers                                                                             # Print route helper methods
rake grape:routes                                                                                   # Print compiled grape routes
rake import:github[token,gitlab_username,project_path]                                              # GitLab | Import | Import a GitHub project - Example: import:github[ToKeN,root,root/blah,my/github_repo]...
rake jira:generate_consumer_key                                                                     # Generate a consumer key for your application
rake jira:generate_public_cert                                                                      # Run the system call to generate a RSA public certificate
rake log:clear                                                                                      # Truncates all/specified *.log files in log/ to zero bytes (specify which logs with LOGS=test,development)
rake metrics:setup_common_metrics                                                                   # GitLab | Metrics | Setup common metrics
rake middleware                                                                                     # Prints out your Rack middleware stack
rake migrate_iids                                                                                   # GitLab | Build internal ids for issues and merge requests
rake migration_fix_15_11                                                                            # db | migration_fix_15_11
rake postgresql_md5_hash                                                                            # GitLab | Generate PostgreSQL Password Hash
rake raven:test[dsn]                                                                                # Send a test event to the remote Sentry server
rake restart                                                                                        # Restart app by touching tmp/restart.txt
rake secret                                                                                         # Generate a cryptographically secure secret key (this is typically used to generate a secret for cookie ...
rake setup                                                                                          # GitLab | Setup gitlab db
rake stats                                                                                          # Report code statistics (KLOCs, etc) from the application or engine
rake tanuki_emoji:aliases                                                                           # Generates Emoji aliases fixtures
rake tanuki_emoji:digests                                                                           # Generates Emoji SHA256 digests
rake tanuki_emoji:import                                                                            # Import emoji assets from TanukiEmoji to versioned folder
rake test                                                                                           # GitLab | List rake tasks for tests
rake test:all                                                                                       # Runs all tests, including system tests
rake test:db                                                                                        # Run tests quickly, but also reset db
rake test:system                                                                                    # Run system tests only
rake time:zones[country_or_offset]                                                                  # List all time zones, list by two-letter country code (`bin/rails time:zones[US]`), or list by UTC offse...
rake tmp:clear                                                                                      # Clear cache, socket and screenshot files from tmp/ (narrow w/ tmp:cache:clear, tmp:sockets:clear, tmp:s...
rake tmp:create                                                                                     # Creates tmp directories for cache, sockets, and pids
rake tokens:reset_all_email                                                                         # Reset all GitLab incoming email tokens
rake tokens:reset_all_feed                                                                          # Reset all GitLab feed tokens
rake tw:codeowners                                                                                  # Generates a list of codeowners for documentation pages
rake yarn                                                                                           # Install Node dependencies with Yarn
rake yarn:available                                                                                 # Ensure Yarn is installed
rake yarn:check                                                                                     # Ensure Node dependencies are installed
rake yarn:clobber                                                                                   # Remove Node dependencies
rake yarn:install                                                                                   # Install Node dependencies with Yarn / Install all JavaScript dependencies as specified via Yarn
rake zeitwerk:check                                                                                 # Checks project structure for Zeitwerk compatibility

Apr

2024

How to catch and analyze Inventree Python API HTTPErrors

This example shows how to differentiate between a Part with this Part Number already exists error and other errors for inventree Part.create(). Any other error is simply re-raised. See Inventree Python API: How to create a new part

try:
    part = Part.create(api, {
        'name': product.mpn,
        'description': product.description,
        'category': get_part_category_by_pathstring(api, "Elektronik-Komponenten").pk,
    })
except requests.exceptions.HTTPError as ex:
    errmsg, = ex.args
    # NOTE: errmsg is a dictionary with the following keys
    # errmsg["detail"] == "Error occurred during API request"
    # errmsg["data"] == {'name': 'VHR-4N-BK', 'description': '...', ...}
    # errmsg["params"] == {'format': 'json'}
    # errmsg["body"] == "{'non_field_erro   rs': ['Part with this Part Number already exists.']}""
    
    body = json.loads(errmsg.get("body", []))
    non_field_errors = body.get("non_field_errors", [])
    # If there is a non field error and it contains "Part with this ... already exists",
    if len(non_field_errors) > 0 and re.match(r"Part with this .* already exists", non_field_errors[0]):
        print("This part already exists")
    else:
        raise ex

Apr

2024

http: How to get redirect location in Python

response = httpx.post("https://my.domain/api/redirect") 
# Get redirect location
if response.has_redirect_location:
    redirect_location = response.headers['Location']
else:
    redirect_location = None

Apr

2024

How to ‘wp package install’ in wpcli docker container

Using docker-compose with a config such as

# ...
  wpcli:
    depends_on:
      - wordpress
    image: wordpress:cli
    user: 1000:1000
    command: tail -f /dev/null
    volumes:
      - ./wordpress:/var/www/html
    environment:
        - WORDPRESS_DB_HOST=mariadb:3306
        - WORDPRESS_DB_USER=wordpress
        - WORDPRESS_DB_NAME=wordpress
        - WORDPRESS_DB_PASSWORD=${MARIADB_PASSWORD}

You need to install the package in the container as root to avoid

Error: Composer directory '/.wp-cli/packages' for packages couldn't be created: mkdir(): Permission denied

Use the following command to install a package

docker-compose exec -u root wpcli wp package install wp-cli/profile-command:@stable --allow-root

Apr

2024

How to change admin password using wpcli on docker-compose

docker-compose exec wpcli wp user update admin --user_pass='ee7saeNg9Moh5shoodocaixaingoov'

This works, for example with a docker-compose wpcli config like:

wpcli:
    depends_on:
      - wordpress
    image: wordpress:cli
    user: 1000:1000
    command: tail -f /dev/null
    volumes:
      - ./wordpress:/var/www/html
    environment:
        - WORDPRESS_DB_HOST=mariadb:3306
        - WORDPRESS_DB_USER=wordpress
        - WORDPRESS_DB_NAME=wordpress
        - WORDPRESS_DB_PASSWORD=${MARIADB_PASSWORD}

Apr

2024

How to identify which docker container is consuming 100% CPU

Open htop and press F6 and select PERCENT_CPU. Repeat until you’ve sorted descending by CPU consumption
Now identify the PID (process ID) of the offending process consuming 100% CPU

In our example, the offending PID is 8599
Now run this command to find a list of the running docker container by PID:

docker inspect -f '{{.State.Pid}}, {{.Name}}' $(docker ps -q)

Now identify the line with the PID, e.g.

8599, /wordpress-myblog-btronik_mariadb_1

Now you know which container is consuming 100% CPU.

Apr

2024

How to find list of docker containers with PID (process ID)

The following command will show you a list of all running docker containers with their PIDs

docker inspect -f '{{.State.Pid}}, {{.Name}}' $(docker ps -q)

Example output:

9046, /wordpress-myblog-btronik_wpcli_1
8848, /wordpress-myblog-btronik_wordpress_1
8795, /wordpress-myblog-btronik_phpmyadmin_1
8599, /wordpress-myblog-btronik_mariadb_1
3627, /wordpress-secondpage_wpcli_1
3505, /wordpress-mywebsite_wpcli_1
3172, /wordpress-secondpage_wordpress_1
2907, /wordpress-mywebsite_wordpress_1
2646, /wordpress-secondpage_mariadb_1
2518, /webmail_roundcube_1
2323, /wordpress-mywebsite_mariadb_1
2303, /redmine-mywebsite_mariadb_1

Apr

2024

JLCPCB PCB assembly: List of known “Standard only” parts

For some parts, JLCPCB doesn’t support Economic PCBA but charges 25$ per assembly side for Standard PCBA.

You can find out which parts are Standard only by checking the JLCPCB component list. Standard only parts are marked Standard only directly after the part number column while parts supporting economic PCBA are not marked at all.

Know parts include, according to my own research (this list is non-exhaustive and I don’t take any responsibility for correctness, beyond what’s legally required):

Many ESP32 modules
- …but not all RF modules – e.g. this one is available in economic PCB
- Some ESP32 modules such as ESP32-SOLO-1C is available in economic mode)
Some U-BLOX Bluetooth modules (but not all U-BLOX modules)
Some ESP32 chips like this one
WS2812 LEDs in 5050 package
Round-top SMD LEDs such as this UV LED
Some chip-scale package chips such as this EFM8 or this flash (but not all CSP chips)
Some large BGA microcontrollers with 100s of pins such as this one

Apr

2024

Angular: How to format number without any decimals using ‘number’ pipe

In Angular, you can use the number pipe to format a number without decimals by specifying '1.0-0' as the format. Here is how you can do it:

<div class="value">{{myvalue | number:'1.0-0'}}</div>

In order for this to work, you need to import CommonModule:

imports: [
  CommonModule
],

Apr

2024

Angular HttpClient: Return both Blob and response Headers

export interface BlobAndHeaders {
  blob: Blob;
  headers: Headers;
}

myApi(arg: string): Observable<BlobAndHeaders> {
  return this.http.get<BlobAndHeaders>(`${this.baseURL}/api/myapi`, {
    observe: 'response',
    responseType: 'blob' as 'json',
    params: {arg: arg},
  }).pipe(map(
    response => {
      if(response.body === null) {
        throw new Error('No data received');
      }
      return {"blob": response.body, "headers": response.headers};
    }
  ));
}

Apr

2024

How to fix Python TypeError: lru_cache() got an unexpected keyword argument ‘max_size

Problem:

In Python, you’re using code such as

from functools import lru_cache

@lru_cache(max_size=128)
def myfunc(arg: str):
   # ...
   return arg + "x"

but you see an error message such as

Traceback (most recent call last):
  File "/dev/shm/test.py", line 3, in <module>
    @lru_cache(max_size=128)
TypeError: lru_cache() got an unexpected keyword argument 'max_size'

Solution:

You’ve mis-spelled the argument to @lru_cache: You wrote max_size with underscore whereas it actually is maxsize without underscore.

Apr

2024

Which distributors can JLCPCB source components from?

Please note: I can’t guarantee any of this, this is just according to my own research in April 2024

JLCPCB can order components from the following distributors:

DigiKey
Mouser
TTI
Chip1stop
Verical
Sager
Arrow
Element14 UK
Future
Powell
Avnet
Element
Onlinecomponents
Rutronik
CoreStaff
TME

Apr

2024

JLCPCB PCBA assembly price overview

This post summarizes JLCPCB fees. Please note that this is according to my own research in April 2024 and I can’t guarantee any of these prices or price structures in any way.

For reference, also see the official price list which lists most of these costs as well: https://jlcpcb.com/help/article/98-pcb-assembly-faqs

Economic vs standard PCBA

Economic PCBA: One-side only, 0$
Standard PCBA, 25$ for one-sided PCBA, 50$ for two-sided PCBA

Some parts such as ESP32 modules or WS2812-style LEDs force you to select standard PCBA.

See JLCPCB PCB assembly: List of known Standard only parts for more details.

Basic vs extended components

Basic components: 0$ per component type
Extended components: 2.74€/3$ per component type

SMT Assembly fees per solder joint

0.0017$ per SMD pin

THT fees / Wave soldering fees

Manual assembly fee: 0.0173$ per THT pin. This is actually a wave soldering fee.
Fixed Hand soldering-fee3.22€/3.50$, this is actually a wave soldering fee with “assembly by hand”

Other fees

Special component fee: For USB-C connectors such as HC-TYPE-C-16P-01A, e.g. €0.1382 per component type (not per individual component)
Stencil: 1,38€/1.50$ fixed fee
Setup fee: 7,37€/8.00$ fixed fee
Panel: 0,00€, unclear how this is computed
Large size: 0.00€, unclear how this is computed
Components: How much your components cost, no additional markup.

Apr

2024

How to iterate rows of a pandas DataFrame with timestamp index

You can use df.iterrows() normally:

import pandas as pd
import numpy as np

# Assuming 'df' is your DataFrame
# For demonstration, let's create a sample DataFrame
dates = pd.date_range('2023-01-01', periods=5)
data = np.random.randn(5, 2)
df = pd.DataFrame(data, index=dates, columns=['A', 'B'])

# Iterating over rows
for index, row in df.iterrows():
    print(f"Index: {index}, A: {row['A']}, B: {row['B']}")

Note that index is a pd.Timestamp.

Apr

2024

How to drop multiple rows from pandas DataFrame by index

import pandas as pd
import numpy as np

# Create a sample DataFrame with datetime index
dates = pd.date_range('2023-01-01', periods=10)
data = np.random.randn(10, 2)
df = pd.DataFrame(data, index=dates, columns=['A', 'B'])

# Timestamps to drop
timestamps_to_drop = [pd.Timestamp('2023-01-03'), pd.Timestamp('2023-01-05'), pd.Timestamp('2023-01-07'),
                      pd.Timestamp('2023-01-08'), pd.Timestamp('2023-01-10')]

# Drop rows
df_filtered = df.drop(index=timestamps_to_drop)

Mar

2024

How to fix “snap refresh”: snap “…” has running apps

Problem:

You want to update your snap, e.g. signal-desktop, but when you try to snap refresh it, you see the following error message:

Error: cannot refresh "signal-desktop": snap "signal-desktop" has running apps (signal-desktop),
        pids: 471050,471137,471138,471189,471204,471248

Solution:

You can’t stop the running apps via snap. Just kill them using killall, for example:

killall signal-desktop && sudo snap refresh signal-desktop

Alternatively, you can kill the PIDs listed in the error message (you need to separate them by space, not by comma):

kill 471050 471137 471138 471189 471204 471248

Mar

2024

How to initialize your KiCAD 8 project on the command line

TL;DR:

Inside the directory where you want to create the project, run

wget -qO- https://raw.githubusercontent.com/ulikoehler/KiCAD-ProcessAutomation/master/InitializeKiCad8Project.sh | bash /dev/stdin MyProject

You should replace MyProject (at the end of the command) with your project name.

Note: This will initialize an empty KiCAD project without any libraries. This is equivalent to creating a new project in KiCAD itself (using the GUI).

Continue reading →

Mar

2024

How to fix pyspice OSError: cannot load library ‘libngspice.so’

Problem:

When trying to run a PySpice program, you see an error message such as

OSError: cannot load library 'libngspice.so': libngspice.so: cannot open shared object file: No such file or directory.  Additionally, ctypes.util.find_library() did not manage to locate a library called 'libngspice.so'

Solution:

Install libngspice, often called libngspice0.

On Ubuntu, install it using

sudo apt -y install libngspice0-dev

You need to install the -dev library since libngspice0 only contains libngspice.so.0 whereas the -dev library contains libngspice.so which is required by pyspice.

How to fix pandas / matplotlib error: raise ValueError(f’Date ordinal {x} converts to {dt}..

Problem:

Solution:

How to fix ESP32 ESP_LOGI(…) error: expected ‘)’ before ‘msg’

Problem:

Solution:

gitlab-rake: How to list tasks

Example output

How to catch and analyze Inventree Python API HTTPErrors

http: How to get redirect location in Python

How to ‘wp package install’ in wpcli docker container

How to change admin password using wpcli on docker-compose

How to identify which docker container is consuming 100% CPU

How to find list of docker containers with PID (process ID)

Example output:

JLCPCB PCB assembly: List of known “Standard only” parts

Angular: How to format number without any decimals using ‘number’ pipe

Angular HttpClient: Return both Blob and response Headers

How to fix Python TypeError: lru_cache() got an unexpected keyword argument ‘max_size

Problem:

Solution:

Which distributors can JLCPCB source components from?

JLCPCB PCBA assembly price overview

Economic vs standard PCBA

Basic vs extended components

SMT Assembly fees per solder joint

THT fees / Wave soldering fees

Other fees

How to iterate rows of a pandas DataFrame with timestamp index

How to drop multiple rows from pandas DataFrame by index

How to fix “snap refresh”: snap “…” has running apps

Problem:

Solution:

How to initialize your KiCAD 8 project on the command line

TL;DR:

How to fix pyspice OSError: cannot load library ‘libngspice.so’

Problem:

Solution:

Search

Categories