Skip to content

Merge to production #1996

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sanason merged 5 commits into from
Mar 9, 2026
Merged

Merge to production #1996

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f575252
Use --wait to ensure each job is queued up and doesn't exhaust quota
pburkholder Nov 10, 2025
3eaffd6
Revert set -e
pburkholder Nov 10, 2025
84f1a88
Catch failures and count 'em
pburkholder Nov 10, 2025
803be79
Merge pull request #1901 from pburkholder/pb/clean-up-cron
sanason Mar 9, 2026
eea739d
Don't validate submissions during deletion. (#1995)
sanason Mar 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 75 additions & 67 deletions .circleci/cron.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,75 +4,83 @@
# a non-zero exit code
set -e

echo "Logging into cloud.gov"
# Set # of failures to 0
F=0

function run_production_tasks() {
# === PRODUCTION environment ===================================================
echo "Running tasks in Production..."

PROD_TASK="cf run-task touchpoints-production-sidekiq-worker --wait -c"
# Users
$PROD_TASK "rake scheduled_jobs:send_one_week_until_inactivation_warning" || F=$((F+=1))
$PROD_TASK "rake scheduled_jobs:send_two_weeks_until_inactivation_warning" || F=$((F+=1))
$PROD_TASK "rake scheduled_jobs:deactivate_inactive_users" || F=$((F+=1))

# Forms
# $PROD_TASK "rake scheduled_jobs:send_daily_notifications" || F=$((F+=1))
# $PROD_TASK "rake scheduled_jobs:send_weekly_notifications" || F=$((F+=1))
# $PROD_TASK "rake scheduled_jobs:check_expiring_forms" || F=$((F+=1))
# $PROD_TASK "rake scheduled_jobs:archive_forms" || F=$((F+=1))
$PROD_TASK "rake scheduled_jobs:notify_form_managers_of_inactive_forms" || F=$((F+=1))
# $PROD_TASK "rake scheduled_jobs:delete_submissions_trash" || F=$((F+=1))
echo "Production tasks have completed."
}


function run_staging_tasks() {
# === STAGING environment ======================================================
echo "Running tasks in Staging..."

STAGING_TASK="cf run-task touchpoints-staging-sidekiq-worker --wait -c"

# Users
$STAGING_TASK "rake scheduled_jobs:send_one_week_until_inactivation_warning" || F=$((F+=1))
$STAGING_TASK "rake scheduled_jobs:send_two_weeks_until_inactivation_warning" || F=$((F+=1))
$STAGING_TASK "rake scheduled_jobs:deactivate_inactive_users" || F=$((F+=1))

# Forms
$STAGING_TASK "rake scheduled_jobs:send_daily_notifications" || F=$((F+=1))
$STAGING_TASK "rake scheduled_jobs:send_weekly_notifications" || F=$((F+=1))
$STAGING_TASK "rake scheduled_jobs:check_expiring_forms" || F=$((F+=1))
$STAGING_TASK "rake scheduled_jobs:archive_forms" || F=$((F+=1))
$STAGING_TASK "rake scheduled_jobs:notify_form_managers_of_inactive_forms" || F=$((F+=1))
# $STAGING_TASK "rake scheduled_jobs:delete_submissions_trash" || F=$((F+=1))

echo "Staging tasks have completed."
}

function run_demo_tasks() {
# === DEMO environment =========================================================
echo "Running tasks in Demo..."
DEMO_TASK="cf run-task touchpoints-demo-sidekiq-worker --wait -c"

# Users
$DEMO_TASK "rake scheduled_jobs:send_one_week_until_inactivation_warning" || F=$((F+=1))
$DEMO_TASK "rake scheduled_jobs:send_two_weeks_until_inactivation_warning" || F=$((F+=1))
$DEMO_TASK "rake scheduled_jobs:deactivate_inactive_users" || F=$((F+=1))

# Forms
$DEMO_TASK "rake scheduled_jobs:send_daily_notifications" || F=$((F+=1))
$DEMO_TASK "rake scheduled_jobs:send_weekly_notifications" || F=$((F+=1))
$DEMO_TASK "rake scheduled_jobs:check_expiring_forms" || F=$((F+=1))
$DEMO_TASK "rake scheduled_jobs:archive_forms" || F=$((F+=1))
$DEMO_TASK "rake scheduled_jobs:notify_form_managers_of_inactive_forms" || F=$((F+=1))
# $DEMO_TASK "rake scheduled_jobs:delete_submissions_trash" || F=$((F+=1))

echo "Demo tasks have completed."
}

echo "Logging into cloud.gov non-prod"
cf login -a $CF_API_ENDPOINT -u $CF_USERNAME -p $CF_PASSWORD -o $CF_ORG -s $CF_SPACE

#
# === STAGING environment ======================================================
#

echo "Running tasks in Staging..."

# Users
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:send_one_week_until_inactivation_warning"
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:send_two_weeks_until_inactivation_warning"
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:deactivate_inactive_users"

# Forms
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:send_daily_notifications"
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:send_weekly_notifications"
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:check_expiring_forms"
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:archive_forms"
cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:notify_form_managers_of_inactive_forms"
# cf run-task touchpoints-staging-sidekiq-worker -c "rake scheduled_jobs:delete_submissions_trash"

echo "Staging tasks have completed."

#
# === DEMO environment =========================================================
#

echo "Running tasks in Demo..."

# Users
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:send_one_week_until_inactivation_warning"
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:send_two_weeks_until_inactivation_warning"
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:deactivate_inactive_users"

# Forms
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:send_daily_notifications"
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:send_weekly_notifications"
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:check_expiring_forms"
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:archive_forms"
cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:notify_form_managers_of_inactive_forms"
# cf run-task touchpoints-demo-sidekiq-worker -c "rake scheduled_jobs:delete_submissions_trash"

echo "Demo tasks have completed."

run_staging_tasks
run_demo_tasks
cf logout

#
# === PRODUCTION environment ===================================================
#

echo "Logging into cloud.gov"
echo "Logging into cloud.gov production environment"
cf login -a $CF_API_ENDPOINT -u $CF_PRODUCTION_SPACE_DEPLOYER_USERNAME -p $CF_PRODUCTION_SPACE_DEPLOYER_PASSWORD -o $CF_ORG -s prod

echo "Running tasks in Production..."

# Users
cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:send_one_week_until_inactivation_warning"
cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:send_two_weeks_until_inactivation_warning"
cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:deactivate_inactive_users"

# Forms
# cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:send_daily_notifications"
# cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:send_weekly_notifications"
# cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:check_expiring_forms"
# cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:archive_forms"
cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:notify_form_managers_of_inactive_forms"
# cf run-task touchpoints-production-sidekiq-worker -c "rake scheduled_jobs:delete_submissions_trash"

echo "Production tasks have completed."

run_production_tasks
cf logout

echo "$0 exiting with failure count: $F"
exit $F
27 changes: 13 additions & 14 deletions app/controllers/admin/submissions_controller.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,31 +186,29 @@ def unmark
def delete
ensure_form_manager(form: @form)

if @submission.update(deleted: true, deleted_at: Time.now)
Event.log_event(Event.names[:response_deleted], 'Submission', @submission.id, "Submission #{@submission.id} deleted at #{DateTime.now}", current_user.id)
else
Rails.logger.warn("Failed to delete submission: #{@submission.errors.full_messages.join(', ')}")
end
Event.log_event(Event.names[:response_deleted], 'Submission', @submission.id, "Submission #{@submission.id} deleted at #{DateTime.now}", current_user.id)
@submission.assign_attributes(deleted: true, deleted_at: Time.now)
@submission.save(validate: false)
end

def destroy
ensure_form_manager(form: @form)

if @submission.update(deleted: true, deleted_at: Time.now)
Event.log_event(Event.names[:response_deleted], 'Submission', @submission.id, "Submission #{@submission.id} deleted at #{DateTime.now}", current_user.id)
respond_to do |format|
format.js { render :destroy }
end
else
Rails.logger.warn("Failed to delete submission: #{@submission.errors.full_messages.join(', ')}")
Event.log_event(Event.names[:response_deleted], 'Submission', @submission.id, "Submission #{@submission.id} deleted at #{DateTime.now}", current_user.id)
@submission.assign_attributes(deleted: true, deleted_at: Time.now)
@submission.save(validate: false)

respond_to do |format|
format.js { render :destroy }
end
end

def undelete
ensure_form_manager(form: @form)

Event.log_event(Event.names[:response_undeleted], 'Submission', @submission.id, "Submission #{@submission.id} undeleted at #{DateTime.now}", current_user.id)
@submission.update(deleted: false, deleted_at: nil)
@submission.assign_attributes(deleted: false, deleted_at: nil)
@submission.save(validate: false)
end

def feed
Expand Down Expand Up @@ -302,7 +300,8 @@ def bulk_update
when 'delete'
submissions.each do |submission|
Event.log_event(Event.names[:response_deleted], 'Submission', submission.id, "Submission #{submission.id} deleted at #{DateTime.now}", current_user.id)
submission.update(deleted: true, deleted_at: Time.now)
submission.assign_attributes(deleted: true, deleted_at: Time.now)
submission.save(validate: false)
end
flash[:notice] = "#{view_context.pluralize(submissions.count, 'Submission')} deleted."
else
Expand Down
4 changes: 2 additions & 2 deletions app/models/submission.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ class Submission < ApplicationRecord
belongs_to :form, counter_cache: :response_count
attr_accessor :fba_directive # for SPAM capture

validate :validate_custom_form
validate :validate_answers, on: :create
validates :uuid, uniqueness: true

before_create :set_uuid
Expand Down Expand Up @@ -47,7 +47,7 @@ class Submission < ApplicationRecord
end

# Validate each submitted field against its question type
def validate_custom_form
def validate_answers
# Isolate questions that were answered
answered_questions = attributes.select { |_key, value| value.present? }

Expand Down
50 changes: 31 additions & 19 deletions spec/controllers/admin/submissions_controller_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,28 +9,27 @@
let!(:user_role) { FactoryBot.create(:user_role, user: admin, form:, role: UserRole::Role::FormManager) }
let!(:questions) {
FactoryBot.create(:question,
form:,
question_type: 'text_field',
form_section: form.form_sections.first,
answer_field: 'answer_02',
position: 2,
text: 'Two'
form:,
question_type: 'text_field',
form_section: form.form_sections.first,
answer_field: 'answer_02',
position: 2,
text: 'Two',
)
FactoryBot.create(:question,
form:,
question_type: 'text_field',
form_section: form.form_sections.first,
answer_field: 'answer_03',
position: 3,
text: 'Three'
form:,
question_type: 'text_field',
form_section: form.form_sections.first,
answer_field: 'answer_03',
position: 3,
text: 'Three',
)
FactoryBot.create(:question,
form:,
question_type: 'text_field',
form_section: form.form_sections.first,
answer_field: 'answer_04',
position: 4,
text: 'Four'
:email,
form:,
form_section: form.form_sections.first,
answer_field: 'answer_04',
position: 4,
)

}
Expand All @@ -47,10 +46,11 @@

let(:invalid_attributes) do
{
form_id: form.id,
answer_01: nil,
answer_02: 'James',
answer_03: 'Madison',
answer_04: nil,
answer_04: 'not_an_email',
}
end

Expand Down Expand Up @@ -99,6 +99,18 @@
end.to change { Submission.active.count }.by(-1)
end

it 'deletes the requested submission even if it has invalid answers' do
expect { Submission.create! invalid_attributes }.to raise_error(ActiveRecord::RecordInvalid)

submission = Submission.create! valid_attributes
submission.assign_attributes(invalid_attributes)
submission.save(validate: false)

expect do
delete :delete, params: { id: submission.to_param, form_id: form.short_uuid }, session: valid_session, format: :js
end.to change { Submission.active.count }.by(-1)
end

it 'redirects to the submissions list' do
submission = Submission.create! valid_attributes
delete :destroy, params: { id: submission.to_param, form_id: form.short_uuid }, session: valid_session, format: :js
Expand Down
4 changes: 2 additions & 2 deletions spec/features/admin/submissions_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,9 @@
visit admin_form_submission_path(form, submission)
end

it 'try to update a submission that has had its question validations changed' do
it 'can update status of a submission that has had its question validations changed' do
click_on("Acknowledge")
expect(page).to have_content("Response could not be updated.")
expect(page).to have_content("Response was successfully updated.")
end
end
end
Expand Down
Loading